This adds computed properties to readable and writable streams to
allow access to the readable buffer, the writable buffer, and flow
state without accessing the readable or writable state.
These are the only uses of readable and writable state in the docs
so adding these work arounds allows them to be removed from the docs.
This also updates net, http_client and http_server to use the new
methods instead of manipulating readable and writable state directly.
See: https://github.com/nodejs/node/issues/445
PR-URL: https://github.com/nodejs/node/pull/12855
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
With the new Ignition+TurboFan pipeline, the instanceof fast-path can be
missed if the right-hand side needs a TDZ check, i.e. is const declared
on a surrounding scope. This doesn't apply to Node 8 at this point,
where it's at V8 5.8, but it applies as soon as V8 5.9 rolls. There's
work going on in Ignition (and TurboFan) to optimize those TDZ checks
properly, but those changes will land in V8 6.1, so might not end up in
Node 8.
One way to work-around this in Node core for now is to use var instead
of const for those right-hand sides for instanceof for now, especially
Buffer in case of streams. This is not beautiful, but proper ducktape.
Improves readable-bigread.js by ~23% with Node LKGR.
PR-URL: https://github.com/nodejs/node/pull/13403
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Calvin Metcalf <calvin.metcalf@gmail.com>
Adds destroy() and _destroy() methods to Readable, Writable, Duplex
and Transform. It also standardizes the behavior and the implementation
of destroy(), which has been inconsistent in userland and core.
This PR also updates all the subsystems of core to use the new
destroy().
PR-URL: https://github.com/nodejs/node/pull/12925
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Calvin Metcalf <calvin.metcalf@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Fix the uncommon situation when a readable stream is piped twice into
the same destination stream, and then unpiped once.
Previously, the `unpipe` event handlers weren’t able to tell whether
they were corresponding to the “right” conceptual pipe that was being
removed; this fixes this by adding a counter to the `unpipe` event
handler and only removing a single piping destination at most.
Fixes: https://github.com/nodejs/node/issues/12718
PR-URL: https://github.com/nodejs/node/pull/12746
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Currently when the destination emits an 'error', 'finish' or 'close'
event the pipe calls unpipe to emit 'unpipe' and trigger the clean up
of all it's listeners.
When the source emits an 'end' event without {end: false} it calls
end() on the destination leading it to emit a 'close', this will again
lead to the pipe calling unpipe. However the source emitting an 'end'
event along side {end: false} is the only time the cleanup gets ran
directly without unpipe being called. This fixes that so the 'unpipe'
event does get emitted and cleanup in turn gets ran by that event.
Fixes: https://github.com/nodejs/node/issues/11837
PR-URL: https://github.com/nodejs/node/pull/11876
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Improve message when tranform._transform() method is not implemented
Improve error message when Readable._read() is not implemented
Remove extra word in err msg when Writable._write() when not implemented
Remove extra word in err msg when Transform._transform() when not implemented
PR-URL: https://github.com/nodejs/node/pull/8801
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
Guard against the call to write() inside pipe's ondata pushing more data
back onto the Readable, thus causing ondata to be called again.
This is fine but results in awaitDrain being increased more than once.
The problem with that is when the destination does drain, only a single
'drain' event is emitted, so awaitDrain in this case will never reach
zero and we end up with a permanently paused stream.
Fixes: https://github.com/nodejs/node/issues/7278
PR-URL: https://github.com/nodejs/node/pull/7292
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
read() performance is improved most by switching from an array to
a linked list for storing buffered data. However, other changes that
also contribute include: making some hot functions inlinable, faster
read() argument checking, and misc code rearrangement to avoid
unnecessary code execution.
PR-URL: https://github.com/nodejs/node/pull/7077
Reviewed-By: Calvin Metcalf <calvin.metcalf@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reset the `readableState.awaitDrain` counter after manual calls to
`.resume()`.
What might happen otherwise is that a slow consumer at the end of the
pipe could end up stalling the piping in the following scenario:
1. The writable stream indicates that its buffer is full.
2. This leads the readable stream to `pause()` and increase its
`awaitDrain` counter, which will be decreased by the writable’s next
`drain` event.
3. Something calls `.resume()` manually.
4. The readable continues to pipe to the writable, but once again
the writable stream indicates that the buffer is full.
5. The `awaitDrain` counter is thus increased again, but since it has
now been increased twice for a single piping destination, the next
`drain` event will not be able to reset `awaitDrain` to zero.
6. The pipe is stalled and no data is passed along anymore.
The solution in this commit is to reset the `awaitDrain` counter to
zero when `resume()` is called.
Fixes: https://github.com/nodejs/node/issues/7159
PR-URL: https://github.com/nodejs/node/pull/7160
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
A handful of modules (including readable-streams) make
inappropriate use of the internal _events property. One
such use is to prepend an event listener to the front
of the array of listeners.
This adds EE.prototype.prependListener() and
EE.prototype.prependOnceListener() methods to add handlers
to the *front* of the listener array.
Doc update and test case is included.
Fixes: https://github.com/nodejs/node/issues/1817
PR-URL: https://github.com/nodejs/node/pull/6032
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Readable.resume() schedules the resume operation onto the next tick,
whereas pause() has immediate effect. This means that in a sequence
stream.resume();
stream.pause();
.. the 'pause' event will be triggered before the resume operation
is performed.
For process.stdin, we are relying on the 'pause' event to stop reading
on the underlying handle. This fix ensures that reads are started and
stopped in the same order as resume() and pause() are called.
PR-URL: https://github.com/nodejs/node/pull/5776
Reviewed-By: cjihrig - Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: jasnell - James M Snell <jasnell@gmail.com>
Several changes:
* Soft-Deprecate Buffer() constructors
* Add `Buffer.from()`, `Buffer.alloc()`, and `Buffer.allocUnsafe()`
* Add `--zero-fill-buffers` command line option
* Add byteOffset and length to `new Buffer(arrayBuffer)` constructor
* buffer.fill('') previously had no effect, now zero-fills
* Update the docs
PR-URL: https://github.com/nodejs/node/pull/4682
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Before this commit, it was possible to push a partial character
to a readable stream where it was decoded as an empty string and
then added to the internal buffer. This caused the stream to not
emit any data, even when the rest of the character bytes were pushed
separately, because of a non-zero length check of the first chunk in
the internal buffer.
Fixes: https://github.com/nodejs/node/issues/5223
PR-URL: https://github.com/nodejs/node/pull/5226
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
`lib/_stream_readable.js` contained three instances of `var`
declarations occurring twice in the same scope. Refactored to `const` or
`let` as appropriate.
PR-URL: https://github.com/nodejs/node/pull/4816
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
ReadableState has the resumeScheduled property that helps determine if
a stream should be resumed. It was not assigned in the constructor.
When stream.resume is called on a readable stream that is not flowing,
it is set to true. This changes the property map of the ReadableState
which can cause a deopt in onEofChunk and needMoreData.
PR-URL: https://github.com/nodejs/node/pull/4761
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
