fs: introduce opendir() and fs.Dir

This adds long-requested methods for asynchronously interacting and
iterating through directory entries by using `uv_fs_opendir`,
`uv_fs_readdir`, and `uv_fs_closedir`.

`fs.opendir()` and friends return an `fs.Dir`, which contains methods
for doing reads and cleanup. `fs.Dir` also has the async iterator
symbol exposed.

The `read()` method and friends only return `fs.Dirent`s for this API.
Having a entry type or doing a `stat` call is deemed to be necessary in
the majority of cases, so just returning dirents seems like the logical
choice for a new api.

Reading when there are no more entries returns `null` instead of a
dirent. However the async iterator hides that (and does automatic
cleanup).

The code lives in separate files from the rest of fs, this is done
partially to prevent over-pollution of those (already very large)
files, but also in the case of js allows loading into `fsPromises`.

Due to async_hooks, this introduces a new handle type of `DIRHANDLE`.

This PR does not attempt to make complete optimization of
this feature. Notable future improvements include:
- Moving promise work into C++ land like FileHandle.
- Possibly adding `readv()` to do multi-entry directory reads.
- Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation.

Refs: https://github.com/nodejs/node-v0.x-archive/issues/388
Refs: https://github.com/nodejs/node/issues/583
Refs: https://github.com/libuv/libuv/pull/2057

PR-URL: https://github.com/nodejs/node/pull/29349
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: David Carlier <devnexen@gmail.com>
This commit is contained in:
Jeremiah Senkpiel
2019-08-27 17:14:27 -07:00
parent 064e111515
commit cbd8d715b2
19 changed files with 1165 additions and 119 deletions

View File

@@ -826,6 +826,11 @@ A signing `key` was not provided to the [`sign.sign()`][] method.
[`crypto.timingSafeEqual()`][] was called with `Buffer`, `TypedArray`, or
`DataView` arguments of different lengths.
<a id="ERR_DIR_CLOSED"></a>
### ERR_DIR_CLOSED
The [`fs.Dir`][] was previously closed.
<a id="ERR_DNS_SET_SERVERS_FAILED"></a>
### ERR_DNS_SET_SERVERS_FAILED
@@ -2388,6 +2393,7 @@ such as `process.stdout.on('data')`.
[`dgram.disconnect()`]: dgram.html#dgram_socket_disconnect
[`dgram.remoteAddress()`]: dgram.html#dgram_socket_remoteaddress
[`errno`(3) man page]: http://man7.org/linux/man-pages/man3/errno.3.html
[`fs.Dir`]: fs.html#fs_class_fs_dir
[`fs.readFileSync`]: fs.html#fs_fs_readfilesync_path_options
[`fs.readdir`]: fs.html#fs_fs_readdir_path_options_callback
[`fs.symlink()`]: fs.html#fs_fs_symlink_target_path_type_callback

View File

@@ -284,13 +284,148 @@ synchronous use libuv's threadpool, which can have surprising and negative
performance implications for some applications. See the
[`UV_THREADPOOL_SIZE`][] documentation for more information.
## Class fs.Dir
<!-- YAML
added: REPLACEME
-->
A class representing a directory stream.
Created by [`fs.opendir()`][], [`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
Example using async interation:
```js
const fs = require('fs');
async function print(path) {
const dir = await fs.promises.opendir(path);
for await (const dirent of dir) {
console.log(dirent.name);
}
}
print('./').catch(console.error);
```
### dir.path
<!-- YAML
added: REPLACEME
-->
* {string}
The read-only path of this directory as was provided to [`fs.opendir()`][],
[`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
### dir.close()
<!-- YAML
added: REPLACEME
-->
* Returns: {Promise}
Asynchronously close the directory's underlying resource handle.
Subsequent reads will result in errors.
A `Promise` is returned that will be resolved after the resource has been
closed.
### dir.close(callback)
<!-- YAML
added: REPLACEME
-->
* `callback` {Function}
* `err` {Error}
Asynchronously close the directory's underlying resource handle.
Subsequent reads will result in errors.
The `callback` will be called after the resource handle has been closed.
### dir.closeSync()
<!-- YAML
added: REPLACEME
-->
Synchronously close the directory's underlying resource handle.
Subsequent reads will result in errors.
### dir.read([options])
<!-- YAML
added: REPLACEME
-->
* `options` {Object}
* `encoding` {string|null} **Default:** `'utf8'`
* Returns: {Promise} containing {fs.Dirent}
Asynchronously read the next directory entry via readdir(3) as an
[`fs.Dirent`][].
A `Promise` is returned that will be resolved with a [Dirent][] after the read
is completed.
_Directory entries returned by this function are in no particular order as
provided by the operating system's underlying directory mechanisms._
### dir.read([options, ]callback)
<!-- YAML
added: REPLACEME
-->
* `options` {Object}
* `encoding` {string|null} **Default:** `'utf8'`
* `callback` {Function}
* `err` {Error}
* `dirent` {fs.Dirent}
Asynchronously read the next directory entry via readdir(3) as an
[`fs.Dirent`][].
The `callback` will be called with a [Dirent][] after the read is completed.
The `encoding` option sets the encoding of the `name` in the `dirent`.
_Directory entries returned by this function are in no particular order as
provided by the operating system's underlying directory mechanisms._
### dir.readSync([options])
<!-- YAML
added: REPLACEME
-->
* `options` {Object}
* `encoding` {string|null} **Default:** `'utf8'`
* Returns: {fs.Dirent}
Synchronously read the next directory entry via readdir(3) as an
[`fs.Dirent`][].
The `encoding` option sets the encoding of the `name` in the `dirent`.
_Directory entries returned by this function are in no particular order as
provided by the operating system's underlying directory mechanisms._
### dir\[Symbol.asyncIterator\]()
<!-- YAML
added: REPLACEME
-->
* Returns: {AsyncIterator} to fully iterate over all entries in the directory.
_Directory entries returned by this iterator are in no particular order as
provided by the operating system's underlying directory mechanisms._
## Class: fs.Dirent
<!-- YAML
added: v10.10.0
-->
When [`fs.readdir()`][] or [`fs.readdirSync()`][] is called with the
`withFileTypes` option set to `true`, the resulting array is filled with
A representation of a directory entry, as returned by reading from an [`fs.Dir`][].
Additionally, when [`fs.readdir()`][] or [`fs.readdirSync()`][] is called with
the `withFileTypes` option set to `true`, the resulting array is filled with
`fs.Dirent` objects, rather than strings or `Buffers`.
### dirent.isBlockDevice()
@@ -2505,6 +2640,46 @@ Returns an integer representing the file descriptor.
For detailed information, see the documentation of the asynchronous version of
this API: [`fs.open()`][].
## fs.opendir(path[, options], callback)
<!-- YAML
added: REPLACEME
-->
* `path` {string|Buffer|URL}
* `options` {Object}
* `encoding` {string|null} **Default:** `'utf8'`
* `callback` {Function}
* `err` {Error}
* `dir` {fs.Dir}
Asynchronously open a directory. See opendir(3).
Creates an [`fs.Dir`][], which contains all further functions for reading from
and cleaning up the directory.
The `encoding` option sets the encoding for the `path` while opening the
directory and subsequent read operations (unless otherwise overriden during
reads from the directory).
## fs.opendirSync(path[, options])
<!-- YAML
added: REPLACEME
-->
* `path` {string|Buffer|URL}
* `options` {Object}
* `encoding` {string|null} **Default:** `'utf8'`
* Returns: {fs.Dir}
Synchronously open a directory. See opendir(3).
Creates an [`fs.Dir`][], which contains all further functions for reading from
and cleaning up the directory.
The `encoding` option sets the encoding for the `path` while opening the
directory and subsequent read operations (unless otherwise overriden during
reads from the directory).
## fs.read(fd, buffer, offset, length, position, callback)
<!-- YAML
added: v0.0.2
@@ -4644,6 +4819,39 @@ by [Naming Files, Paths, and Namespaces][]. Under NTFS, if the filename contains
a colon, Node.js will open a file system stream, as described by
[this MSDN page][MSDN-Using-Streams].
## fsPromises.opendir(path[, options])
<!-- YAML
added: REPLACEME
-->
* `path` {string|Buffer|URL}
* `options` {Object}
* `encoding` {string|null} **Default:** `'utf8'`
* Returns: {Promise} containing {fs.Dir}
Asynchronously open a directory. See opendir(3).
Creates an [`fs.Dir`][], which contains all further functions for reading from
and cleaning up the directory.
The `encoding` option sets the encoding for the `path` while opening the
directory and subsequent read operations (unless otherwise overriden during
reads from the directory).
Example using async interation:
```js
const fs = require('fs');
async function print(path) {
const dir = await fs.promises.opendir(path);
for await (const dirent of dir) {
console.log(dirent.name);
}
}
print('./').catch(console.error);
```
### fsPromises.readdir(path[, options])
<!-- YAML
added: v10.0.0
@@ -5253,6 +5461,7 @@ the file contents.
[`UV_THREADPOOL_SIZE`]: cli.html#cli_uv_threadpool_size_size
[`WriteStream`]: #fs_class_fs_writestream
[`event ports`]: https://illumos.org/man/port_create
[`fs.Dir`]: #fs_class_fs_dir
[`fs.Dirent`]: #fs_class_fs_dirent
[`fs.FSWatcher`]: #fs_class_fs_fswatcher
[`fs.Stats`]: #fs_class_fs_stats
@@ -5269,6 +5478,8 @@ the file contents.
[`fs.mkdir()`]: #fs_fs_mkdir_path_options_callback
[`fs.mkdtemp()`]: #fs_fs_mkdtemp_prefix_options_callback
[`fs.open()`]: #fs_fs_open_path_flags_mode_callback
[`fs.opendir()`]: #fs_fs_opendir_path_options_callback
[`fs.opendirSync()`]: #fs_fs_opendirsync_path_options
[`fs.read()`]: #fs_fs_read_fd_buffer_offset_length_position_callback
[`fs.readFile()`]: #fs_fs_readfile_path_options_callback
[`fs.readFileSync()`]: #fs_fs_readfilesync_path_options
@@ -5284,6 +5495,7 @@ the file contents.
[`fs.write(fd, string...)`]: #fs_fs_write_fd_string_position_encoding_callback
[`fs.writeFile()`]: #fs_fs_writefile_file_data_options_callback
[`fs.writev()`]: #fs_fs_writev_fd_buffers_position_callback
[`fsPromises.opendir()`]: #fs_fspromises_opendir_path_options
[`inotify(7)`]: http://man7.org/linux/man-pages/man7/inotify.7.html
[`kqueue(2)`]: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
[`net.Socket`]: net.html#net_class_net_socket

View File

@@ -64,6 +64,7 @@ const {
getDirents,
getOptions,
getValidatedPath,
handleErrorFromBinding,
nullCheck,
preprocessSymlinkDestination,
Stats,
@@ -79,6 +80,11 @@ const {
validateRmdirOptions,
warnOnNonPortableTemplate
} = require('internal/fs/utils');
const {
Dir,
opendir,
opendirSync
} = require('internal/fs/dir');
const {
CHAR_FORWARD_SLASH,
CHAR_BACKWARD_SLASH,
@@ -122,23 +128,6 @@ function showTruncateDeprecation() {
}
}
function handleErrorFromBinding(ctx) {
if (ctx.errno !== undefined) { // libuv error numbers
const err = uvException(ctx);
// eslint-disable-next-line no-restricted-syntax
Error.captureStackTrace(err, handleErrorFromBinding);
throw err;
}
if (ctx.error !== undefined) { // Errors created in C++ land.
// TODO(joyeecheung): currently, ctx.error are encoding errors
// usually caused by memory problems. We need to figure out proper error
// code(s) for this.
// eslint-disable-next-line no-restricted-syntax
Error.captureStackTrace(ctx.error, handleErrorFromBinding);
throw ctx.error;
}
}
function maybeCallback(cb) {
if (typeof cb === 'function')
return cb;
@@ -1834,7 +1823,6 @@ function createWriteStream(path, options) {
return new WriteStream(path, options);
}
module.exports = fs = {
appendFile,
appendFileSync,
@@ -1880,6 +1868,8 @@ module.exports = fs = {
mkdtempSync,
open,
openSync,
opendir,
opendirSync,
readdir,
readdirSync,
read,
@@ -1913,6 +1903,7 @@ module.exports = fs = {
writeSync,
writev,
writevSync,
Dir,
Dirent,
Stats,

View File

@@ -764,6 +764,7 @@ E('ERR_CRYPTO_SCRYPT_NOT_SUPPORTED', 'Scrypt algorithm not supported', Error);
E('ERR_CRYPTO_SIGN_KEY_REQUIRED', 'No key provided to sign', Error);
E('ERR_CRYPTO_TIMING_SAFE_EQUAL_LENGTH',
'Input buffers must have the same byte length', RangeError);
E('ERR_DIR_CLOSED', 'Directory handle was closed', Error);
E('ERR_DNS_SET_SERVERS_FAILED', 'c-ares failed to set servers: "%s" [%s]',
Error);
E('ERR_DOMAIN_CALLBACK_NOT_AVAILABLE',

201
lib/internal/fs/dir.js Normal file
View File

@@ -0,0 +1,201 @@
'use strict';
const { Object } = primordials;
const pathModule = require('path');
const binding = internalBinding('fs');
const dirBinding = internalBinding('fs_dir');
const {
codes: {
ERR_DIR_CLOSED,
ERR_INVALID_CALLBACK,
ERR_MISSING_ARGS
}
} = require('internal/errors');
const { FSReqCallback } = binding;
const internalUtil = require('internal/util');
const {
getDirent,
getOptions,
getValidatedPath,
handleErrorFromBinding
} = require('internal/fs/utils');
const kDirHandle = Symbol('kDirHandle');
const kDirPath = Symbol('kDirPath');
const kDirClosed = Symbol('kDirClosed');
const kDirOptions = Symbol('kDirOptions');
const kDirReadPromisified = Symbol('kDirReadPromisified');
const kDirClosePromisified = Symbol('kDirClosePromisified');
class Dir {
constructor(handle, path, options) {
if (handle == null) throw new ERR_MISSING_ARGS('handle');
this[kDirHandle] = handle;
this[kDirPath] = path;
this[kDirClosed] = false;
this[kDirOptions] = getOptions(options, {
encoding: 'utf8'
});
this[kDirReadPromisified] = internalUtil.promisify(this.read).bind(this);
this[kDirClosePromisified] = internalUtil.promisify(this.close).bind(this);
}
get path() {
return this[kDirPath];
}
read(options, callback) {
if (this[kDirClosed] === true) {
throw new ERR_DIR_CLOSED();
}
callback = typeof options === 'function' ? options : callback;
if (callback === undefined) {
return this[kDirReadPromisified](options);
} else if (typeof callback !== 'function') {
throw new ERR_INVALID_CALLBACK(callback);
}
options = getOptions(options, this[kDirOptions]);
const req = new FSReqCallback();
req.oncomplete = (err, result) => {
if (err || result === null) {
return callback(err, result);
}
getDirent(this[kDirPath], result[0], result[1], callback);
};
this[kDirHandle].read(
options.encoding,
req
);
}
readSync(options) {
if (this[kDirClosed] === true) {
throw new ERR_DIR_CLOSED();
}
options = getOptions(options, this[kDirOptions]);
const ctx = { path: this[kDirPath] };
const result = this[kDirHandle].read(
options.encoding,
undefined,
ctx
);
handleErrorFromBinding(ctx);
if (result === null) {
return result;
}
return getDirent(this[kDirPath], result[0], result[1]);
}
close(callback) {
if (this[kDirClosed] === true) {
throw new ERR_DIR_CLOSED();
}
if (callback === undefined) {
return this[kDirClosePromisified]();
} else if (typeof callback !== 'function') {
throw new ERR_INVALID_CALLBACK(callback);
}
this[kDirClosed] = true;
const req = new FSReqCallback();
req.oncomplete = callback;
this[kDirHandle].close(req);
}
closeSync() {
if (this[kDirClosed] === true) {
throw new ERR_DIR_CLOSED();
}
this[kDirClosed] = true;
const ctx = { path: this[kDirPath] };
const result = this[kDirHandle].close(undefined, ctx);
handleErrorFromBinding(ctx);
return result;
}
async* entries() {
try {
while (true) {
const result = await this[kDirReadPromisified]();
if (result === null) {
break;
}
yield result;
}
} finally {
await this[kDirClosePromisified]();
}
}
}
Object.defineProperty(Dir.prototype, Symbol.asyncIterator, {
value: Dir.prototype.entries,
enumerable: false,
writable: true,
configurable: true,
});
function opendir(path, options, callback) {
callback = typeof options === 'function' ? options : callback;
if (typeof callback !== 'function') {
throw new ERR_INVALID_CALLBACK(callback);
}
path = getValidatedPath(path);
options = getOptions(options, {
encoding: 'utf8'
});
function opendirCallback(error, handle) {
if (error) {
callback(error);
} else {
callback(null, new Dir(handle, path, options));
}
}
const req = new FSReqCallback();
req.oncomplete = opendirCallback;
dirBinding.opendir(
pathModule.toNamespacedPath(path),
options.encoding,
req
);
}
function opendirSync(path, options) {
path = getValidatedPath(path);
options = getOptions(options, {
encoding: 'utf8'
});
const ctx = { path };
const handle = dirBinding.opendir(
pathModule.toNamespacedPath(path),
options.encoding,
undefined,
ctx
);
handleErrorFromBinding(ctx);
return new Dir(handle, path, options);
}
module.exports = {
Dir,
opendir,
opendirSync
};

View File

@@ -36,6 +36,7 @@ const {
validateRmdirOptions,
warnOnNonPortableTemplate
} = require('internal/fs/utils');
const { opendir } = require('internal/fs/dir');
const {
parseMode,
validateBuffer,
@@ -509,6 +510,7 @@ module.exports = {
access,
copyFile,
open,
opendir: promisify(opendir),
rename,
truncate,
rmdir,

View File

@@ -12,7 +12,8 @@ const {
ERR_INVALID_OPT_VALUE_ENCODING,
ERR_OUT_OF_RANGE
},
hideStackFrames
hideStackFrames,
uvException
} = require('internal/errors');
const {
isArrayBufferView,
@@ -165,19 +166,33 @@ function getDirents(path, [names, types], callback) {
} else {
const len = names.length;
for (i = 0; i < len; i++) {
const type = types[i];
if (type === UV_DIRENT_UNKNOWN) {
const name = names[i];
const stats = lazyLoadFs().lstatSync(pathModule.join(path, name));
names[i] = new DirentFromStats(name, stats);
} else {
names[i] = new Dirent(names[i], types[i]);
}
names[i] = getDirent(path, names[i], types[i]);
}
return names;
}
}
function getDirent(path, name, type, callback) {
if (typeof callback === 'function') {
if (type === UV_DIRENT_UNKNOWN) {
lazyLoadFs().lstat(pathModule.join(path, name), (err, stats) => {
if (err) {
callback(err);
return;
}
callback(null, new DirentFromStats(name, stats));
});
} else {
callback(null, new Dirent(name, type));
}
} else if (type === UV_DIRENT_UNKNOWN) {
const stats = lazyLoadFs().lstatSync(pathModule.join(path, name));
return new DirentFromStats(name, stats);
} else {
return new Dirent(name, type);
}
}
function getOptions(options, defaultOptions) {
if (options === null || options === undefined ||
typeof options === 'function') {
@@ -197,6 +212,23 @@ function getOptions(options, defaultOptions) {
return options;
}
function handleErrorFromBinding(ctx) {
if (ctx.errno !== undefined) { // libuv error numbers
const err = uvException(ctx);
// eslint-disable-next-line no-restricted-syntax
Error.captureStackTrace(err, handleErrorFromBinding);
throw err;
}
if (ctx.error !== undefined) { // Errors created in C++ land.
// TODO(joyeecheung): currently, ctx.error are encoding errors
// usually caused by memory problems. We need to figure out proper error
// code(s) for this.
// eslint-disable-next-line no-restricted-syntax
Error.captureStackTrace(ctx.error, handleErrorFromBinding);
throw ctx.error;
}
}
// Check if the path contains null types if it is a string nor Uint8Array,
// otherwise return silently.
const nullCheck = hideStackFrames((path, propName, throwError = true) => {
@@ -558,9 +590,11 @@ module.exports = {
BigIntStats, // for testing
copyObject,
Dirent,
getDirent,
getDirents,
getOptions,
getValidatedPath,
handleErrorFromBinding,
nullCheck,
preprocessSymlinkDestination,
realpathCacheKey: Symbol('realpathCacheKey'),

View File

@@ -121,6 +121,7 @@
'lib/internal/fixed_queue.js',
'lib/internal/freelist.js',
'lib/internal/freeze_intrinsics.js',
'lib/internal/fs/dir.js',
'lib/internal/fs/promises.js',
'lib/internal/fs/read_file_context.js',
'lib/internal/fs/rimraf.js',
@@ -526,6 +527,7 @@
'src/node_constants.cc',
'src/node_contextify.cc',
'src/node_credentials.cc',
'src/node_dir.cc',
'src/node_domain.cc',
'src/node_env_var.cc',
'src/node_errors.cc',
@@ -606,6 +608,7 @@
'src/node_constants.h',
'src/node_context_data.h',
'src/node_contextify.h',
'src/node_dir.h',
'src/node_errors.h',
'src/node_file.h',
'src/node_http2.h',

View File

@@ -33,6 +33,7 @@ namespace node {
#define NODE_ASYNC_NON_CRYPTO_PROVIDER_TYPES(V) \
V(NONE) \
V(DIRHANDLE) \
V(DNSCHANNEL) \
V(ELDHISTOGRAM) \
V(FILEHANDLE) \

View File

@@ -382,6 +382,7 @@ constexpr size_t kFsStatsBufferLength =
V(async_wrap_ctor_template, v8::FunctionTemplate) \
V(async_wrap_object_ctor_template, v8::FunctionTemplate) \
V(compiled_fn_entry_template, v8::ObjectTemplate) \
V(dir_instance_template, v8::ObjectTemplate) \
V(fd_constructor_template, v8::ObjectTemplate) \
V(fdclose_constructor_template, v8::ObjectTemplate) \
V(filehandlereadwrap_template, v8::ObjectTemplate) \

View File

@@ -51,6 +51,7 @@
V(domain) \
V(errors) \
V(fs) \
V(fs_dir) \
V(fs_event_wrap) \
V(heap_utils) \
V(http2) \

350
src/node_dir.cc Normal file
View File

@@ -0,0 +1,350 @@
#include "node_dir.h"
#include "node_process.h"
#include "util.h"
#include "tracing/trace_event.h"
#include "req_wrap-inl.h"
#include "string_bytes.h"
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <cstring>
#include <cerrno>
#include <climits>
#include <memory>
namespace node {
namespace fs_dir {
using fs::FSReqAfterScope;
using fs::FSReqBase;
using fs::FSReqWrapSync;
using fs::GetReqWrap;
using v8::Array;
using v8::Context;
using v8::Function;
using v8::FunctionCallbackInfo;
using v8::FunctionTemplate;
using v8::HandleScope;
using v8::Integer;
using v8::Isolate;
using v8::Local;
using v8::MaybeLocal;
using v8::Null;
using v8::Object;
using v8::ObjectTemplate;
using v8::String;
using v8::Value;
#define TRACE_NAME(name) "fs_dir.sync." #name
#define GET_TRACE_ENABLED \
(*TRACE_EVENT_API_GET_CATEGORY_GROUP_ENABLED \
(TRACING_CATEGORY_NODE2(fs_dir, sync)) != 0)
#define FS_DIR_SYNC_TRACE_BEGIN(syscall, ...) \
if (GET_TRACE_ENABLED) \
TRACE_EVENT_BEGIN(TRACING_CATEGORY_NODE2(fs_dir, sync), TRACE_NAME(syscall), \
##__VA_ARGS__);
#define FS_DIR_SYNC_TRACE_END(syscall, ...) \
if (GET_TRACE_ENABLED) \
TRACE_EVENT_END(TRACING_CATEGORY_NODE2(fs_dir, sync), TRACE_NAME(syscall), \
##__VA_ARGS__);
DirHandle::DirHandle(Environment* env, Local<Object> obj, uv_dir_t* dir)
: AsyncWrap(env, obj, AsyncWrap::PROVIDER_DIRHANDLE),
dir_(dir) {
MakeWeak();
dir_->nentries = 1;
dir_->dirents = &dirent_;
}
DirHandle* DirHandle::New(Environment* env, uv_dir_t* dir) {
Local<Object> obj;
if (!env->dir_instance_template()
->NewInstance(env->context())
.ToLocal(&obj)) {
return nullptr;
}
return new DirHandle(env, obj, dir);
}
void DirHandle::New(const FunctionCallbackInfo<Value>& args) {
CHECK(args.IsConstructCall());
}
DirHandle::~DirHandle() {
CHECK(!closing_); // We should not be deleting while explicitly closing!
GCClose(); // Close synchronously and emit warning
CHECK(closed_); // We have to be closed at the point
}
// Close the directory handle if it hasn't already been closed. A process
// warning will be emitted using a SetImmediate to avoid calling back to
// JS during GC. If closing the fd fails at this point, a fatal exception
// will crash the process immediately.
inline void DirHandle::GCClose() {
if (closed_) return;
uv_fs_t req;
int ret = uv_fs_closedir(nullptr, &req, dir_, nullptr);
uv_fs_req_cleanup(&req);
closing_ = false;
closed_ = true;
struct err_detail { int ret; };
err_detail detail { ret };
if (ret < 0) {
// Do not unref this
env()->SetImmediate([detail](Environment* env) {
char msg[70];
snprintf(msg, arraysize(msg),
"Closing directory handle on garbage collection failed");
// This exception will end up being fatal for the process because
// it is being thrown from within the SetImmediate handler and
// there is no JS stack to bubble it to. In other words, tearing
// down the process is the only reasonable thing we can do here.
HandleScope handle_scope(env->isolate());
env->ThrowUVException(detail.ret, "close", msg);
});
return;
}
// If the close was successful, we still want to emit a process warning
// to notify that the file descriptor was gc'd. We want to be noisy about
// this because not explicitly closing the DirHandle is a bug.
env()->SetUnrefImmediate([](Environment* env) {
ProcessEmitWarning(env,
"Closing directory handle on garbage collection");
});
}
void AfterClose(uv_fs_t* req) {
FSReqBase* req_wrap = FSReqBase::from_req(req);
FSReqAfterScope after(req_wrap, req);
if (after.Proceed())
req_wrap->Resolve(Undefined(req_wrap->env()->isolate()));
}
void DirHandle::Close(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
const int argc = args.Length();
CHECK_GE(argc, 1);
DirHandle* dir;
ASSIGN_OR_RETURN_UNWRAP(&dir, args.Holder());
dir->closing_ = false;
dir->closed_ = true;
FSReqBase* req_wrap_async = GetReqWrap(env, args[0]);
if (req_wrap_async != nullptr) { // close(req)
AsyncCall(env, req_wrap_async, args, "closedir", UTF8, AfterClose,
uv_fs_closedir, dir->dir());
} else { // close(undefined, ctx)
CHECK_EQ(argc, 2);
FSReqWrapSync req_wrap_sync;
FS_DIR_SYNC_TRACE_BEGIN(closedir);
SyncCall(env, args[1], &req_wrap_sync, "closedir", uv_fs_closedir,
dir->dir());
FS_DIR_SYNC_TRACE_END(closedir);
}
}
void AfterDirReadSingle(uv_fs_t* req) {
FSReqBase* req_wrap = FSReqBase::from_req(req);
FSReqAfterScope after(req_wrap, req);
if (!after.Proceed()) {
return;
}
Environment* env = req_wrap->env();
Isolate* isolate = env->isolate();
Local<Value> error;
if (req->result == 0) {
// Done
Local<Value> done = Null(isolate);
req_wrap->Resolve(done);
return;
}
uv_dir_t* dir = static_cast<uv_dir_t*>(req->ptr);
req->ptr = nullptr;
// Single entries are returned without an array wrapper
const uv_dirent_t& ent = dir->dirents[0];
MaybeLocal<Value> filename =
StringBytes::Encode(isolate,
ent.name,
req_wrap->encoding(),
&error);
if (filename.IsEmpty())
return req_wrap->Reject(error);
Local<Array> result = Array::New(isolate, 2);
result->Set(env->context(),
0,
filename.ToLocalChecked()).FromJust();
result->Set(env->context(),
1,
Integer::New(isolate, ent.type)).FromJust();
req_wrap->Resolve(result);
}
void DirHandle::Read(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
Isolate* isolate = env->isolate();
const int argc = args.Length();
CHECK_GE(argc, 2);
const enum encoding encoding = ParseEncoding(isolate, args[0], UTF8);
DirHandle* dir;
ASSIGN_OR_RETURN_UNWRAP(&dir, args.Holder());
FSReqBase* req_wrap_async = static_cast<FSReqBase*>(GetReqWrap(env, args[1]));
if (req_wrap_async != nullptr) { // dir.read(encoding, req)
AsyncCall(env, req_wrap_async, args, "readdir", encoding,
AfterDirReadSingle, uv_fs_readdir, dir->dir());
} else { // dir.read(encoding, undefined, ctx)
CHECK_EQ(argc, 3);
FSReqWrapSync req_wrap_sync;
FS_DIR_SYNC_TRACE_BEGIN(readdir);
int err = SyncCall(env, args[2], &req_wrap_sync, "readdir", uv_fs_readdir,
dir->dir());
FS_DIR_SYNC_TRACE_END(readdir);
if (err < 0) {
return; // syscall failed, no need to continue, error info is in ctx
}
if (req_wrap_sync.req.result == 0) {
// Done
Local<Value> done = Null(isolate);
args.GetReturnValue().Set(done);
return;
}
CHECK_GE(req_wrap_sync.req.result, 0);
const uv_dirent_t& ent = dir->dir()->dirents[0];
Local<Value> error;
MaybeLocal<Value> filename =
StringBytes::Encode(isolate,
ent.name,
encoding,
&error);
if (filename.IsEmpty()) {
Local<Object> ctx = args[2].As<Object>();
ctx->Set(env->context(), env->error_string(), error).FromJust();
return;
}
Local<Array> result = Array::New(isolate, 2);
result->Set(env->context(),
0,
filename.ToLocalChecked()).FromJust();
result->Set(env->context(),
1,
Integer::New(isolate, ent.type)).FromJust();
args.GetReturnValue().Set(result);
}
}
void AfterOpenDir(uv_fs_t* req) {
FSReqBase* req_wrap = FSReqBase::from_req(req);
FSReqAfterScope after(req_wrap, req);
if (!after.Proceed()) {
return;
}
Environment* env = req_wrap->env();
Local<Value> error;
uv_dir_t* dir = static_cast<uv_dir_t*>(req->ptr);
DirHandle* handle = DirHandle::New(env, dir);
req_wrap->Resolve(handle->object().As<Value>());
}
static void OpenDir(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
Isolate* isolate = env->isolate();
const int argc = args.Length();
CHECK_GE(argc, 3);
BufferValue path(isolate, args[0]);
CHECK_NOT_NULL(*path);
const enum encoding encoding = ParseEncoding(isolate, args[1], UTF8);
FSReqBase* req_wrap_async = static_cast<FSReqBase*>(GetReqWrap(env, args[2]));
if (req_wrap_async != nullptr) { // openDir(path, encoding, req)
AsyncCall(env, req_wrap_async, args, "opendir", encoding, AfterOpenDir,
uv_fs_opendir, *path);
} else { // openDir(path, encoding, undefined, ctx)
CHECK_EQ(argc, 4);
FSReqWrapSync req_wrap_sync;
FS_DIR_SYNC_TRACE_BEGIN(opendir);
int result = SyncCall(env, args[3], &req_wrap_sync, "opendir",
uv_fs_opendir, *path);
FS_DIR_SYNC_TRACE_END(opendir);
if (result < 0) {
return; // syscall failed, no need to continue, error info is in ctx
}
uv_fs_t* req = &req_wrap_sync.req;
uv_dir_t* dir = static_cast<uv_dir_t*>(req->ptr);
DirHandle* handle = DirHandle::New(env, dir);
args.GetReturnValue().Set(handle->object().As<Value>());
}
}
void Initialize(Local<Object> target,
Local<Value> unused,
Local<Context> context,
void* priv) {
Environment* env = Environment::GetCurrent(context);
Isolate* isolate = env->isolate();
env->SetMethod(target, "opendir", OpenDir);
// Create FunctionTemplate for DirHandle
Local<FunctionTemplate> dir = env->NewFunctionTemplate(DirHandle::New);
dir->Inherit(AsyncWrap::GetConstructorTemplate(env));
env->SetProtoMethod(dir, "read", DirHandle::Read);
env->SetProtoMethod(dir, "close", DirHandle::Close);
Local<ObjectTemplate> dirt = dir->InstanceTemplate();
dirt->SetInternalFieldCount(DirHandle::kDirHandleFieldCount);
Local<String> handleString =
FIXED_ONE_BYTE_STRING(isolate, "DirHandle");
dir->SetClassName(handleString);
target
->Set(context, handleString,
dir->GetFunction(env->context()).ToLocalChecked())
.FromJust();
env->set_dir_instance_template(dirt);
}
} // namespace fs_dir
} // end namespace node
NODE_MODULE_CONTEXT_AWARE_INTERNAL(fs_dir, node::fs_dir::Initialize)

60
src/node_dir.h Normal file
View File

@@ -0,0 +1,60 @@
#ifndef SRC_NODE_DIR_H_
#define SRC_NODE_DIR_H_
#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
#include "node_file.h"
#include "node.h"
#include "req_wrap-inl.h"
namespace node {
namespace fs_dir {
// Needed to propagate `uv_dir_t`.
class DirHandle : public AsyncWrap {
public:
static constexpr int kDirHandleFieldCount = 1;
static DirHandle* New(Environment* env, uv_dir_t* dir);
~DirHandle() override;
static void New(const v8::FunctionCallbackInfo<v8::Value>& args);
static void Open(const v8::FunctionCallbackInfo<Value>& args);
static void Read(const v8::FunctionCallbackInfo<Value>& args);
static void Close(const v8::FunctionCallbackInfo<Value>& args);
inline uv_dir_t* dir() { return dir_; }
AsyncWrap* GetAsyncWrap() { return this; }
void MemoryInfo(MemoryTracker* tracker) const override {
tracker->TrackFieldWithSize("dir", sizeof(*dir_));
}
SET_MEMORY_INFO_NAME(DirHandle)
SET_SELF_SIZE(DirHandle)
DirHandle(const DirHandle&) = delete;
DirHandle& operator=(const DirHandle&) = delete;
DirHandle(const DirHandle&&) = delete;
DirHandle& operator=(const DirHandle&&) = delete;
private:
DirHandle(Environment* env, v8::Local<v8::Object> obj, uv_dir_t* dir);
// Synchronous close that emits a warning
void GCClose();
uv_dir_t* dir_;
uv_dirent_t dirent_;
bool closing_ = false;
bool closed_ = false;
};
} // namespace fs_dir
} // namespace node
#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
#endif // SRC_NODE_DIR_H_

View File

@@ -52,11 +52,9 @@ namespace node {
namespace fs {
using v8::Array;
using v8::BigUint64Array;
using v8::Context;
using v8::DontDelete;
using v8::EscapableHandleScope;
using v8::Float64Array;
using v8::Function;
using v8::FunctionCallbackInfo;
using v8::FunctionTemplate;
@@ -678,94 +676,6 @@ void AfterScanDirWithTypes(uv_fs_t* req) {
req_wrap->Resolve(result);
}
// This class is only used on sync fs calls.
// For async calls FSReqCallback is used.
class FSReqWrapSync {
public:
FSReqWrapSync() = default;
~FSReqWrapSync() { uv_fs_req_cleanup(&req); }
uv_fs_t req;
FSReqWrapSync(const FSReqWrapSync&) = delete;
FSReqWrapSync& operator=(const FSReqWrapSync&) = delete;
};
// Returns nullptr if the operation fails from the start.
template <typename Func, typename... Args>
inline FSReqBase* AsyncDestCall(Environment* env,
FSReqBase* req_wrap,
const FunctionCallbackInfo<Value>& args,
const char* syscall, const char* dest, size_t len,
enum encoding enc, uv_fs_cb after, Func fn, Args... fn_args) {
CHECK_NOT_NULL(req_wrap);
req_wrap->Init(syscall, dest, len, enc);
int err = req_wrap->Dispatch(fn, fn_args..., after);
if (err < 0) {
uv_fs_t* uv_req = req_wrap->req();
uv_req->result = err;
uv_req->path = nullptr;
after(uv_req); // after may delete req_wrap if there is an error
req_wrap = nullptr;
} else {
req_wrap->SetReturnValue(args);
}
return req_wrap;
}
// Returns nullptr if the operation fails from the start.
template <typename Func, typename... Args>
inline FSReqBase* AsyncCall(Environment* env,
FSReqBase* req_wrap,
const FunctionCallbackInfo<Value>& args,
const char* syscall, enum encoding enc,
uv_fs_cb after, Func fn, Args... fn_args) {
return AsyncDestCall(env, req_wrap, args,
syscall, nullptr, 0, enc,
after, fn, fn_args...);
}
// Template counterpart of SYNC_CALL, except that it only puts
// the error number and the syscall in the context instead of
// creating an error in the C++ land.
// ctx must be checked using value->IsObject() before being passed.
template <typename Func, typename... Args>
inline int SyncCall(Environment* env, Local<Value> ctx, FSReqWrapSync* req_wrap,
const char* syscall, Func fn, Args... args) {
env->PrintSyncTrace();
int err = fn(env->event_loop(), &(req_wrap->req), args..., nullptr);
if (err < 0) {
Local<Context> context = env->context();
Local<Object> ctx_obj = ctx.As<Object>();
Isolate* isolate = env->isolate();
ctx_obj->Set(context,
env->errno_string(),
Integer::New(isolate, err)).Check();
ctx_obj->Set(context,
env->syscall_string(),
OneByteString(isolate, syscall)).Check();
}
return err;
}
// TODO(addaleax): Currently, callers check the return value and assume
// that nullptr indicates a synchronous call, rather than a failure.
// Failure conditions should be disambiguated and handled appropriately.
inline FSReqBase* GetReqWrap(Environment* env, Local<Value> value,
bool use_bigint = false) {
if (value->IsObject()) {
return Unwrap<FSReqBase>(value.As<Object>());
} else if (value->StrictEquals(env->fs_use_promises_symbol())) {
if (use_bigint) {
return FSReqPromise<AliasedBigUint64Array>::New(env, use_bigint);
} else {
return FSReqPromise<AliasedFloat64Array>::New(env, use_bigint);
}
}
return nullptr;
}
void Access(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
Isolate* isolate = env->isolate();

View File

@@ -4,7 +4,9 @@
#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
#include "node.h"
#include "aliased_buffer.h"
#include "stream_base.h"
#include "memory_tracker-inl.h"
#include "req_wrap-inl.h"
#include <iostream>
@@ -450,6 +452,93 @@ int MKDirpSync(uv_loop_t* loop,
const std::string& path,
int mode,
uv_fs_cb cb = nullptr);
class FSReqWrapSync {
public:
FSReqWrapSync() = default;
~FSReqWrapSync() { uv_fs_req_cleanup(&req); }
uv_fs_t req;
FSReqWrapSync(const FSReqWrapSync&) = delete;
FSReqWrapSync& operator=(const FSReqWrapSync&) = delete;
};
// TODO(addaleax): Currently, callers check the return value and assume
// that nullptr indicates a synchronous call, rather than a failure.
// Failure conditions should be disambiguated and handled appropriately.
inline FSReqBase* GetReqWrap(Environment* env, v8::Local<v8::Value> value,
bool use_bigint = false) {
if (value->IsObject()) {
return Unwrap<FSReqBase>(value.As<Object>());
} else if (value->StrictEquals(env->fs_use_promises_symbol())) {
if (use_bigint) {
return FSReqPromise<AliasedBigUint64Array>::New(env, use_bigint);
} else {
return FSReqPromise<AliasedFloat64Array>::New(env, use_bigint);
}
}
return nullptr;
}
// Returns nullptr if the operation fails from the start.
template <typename Func, typename... Args>
inline FSReqBase* AsyncDestCall(Environment* env, FSReqBase* req_wrap,
const v8::FunctionCallbackInfo<Value>& args,
const char* syscall, const char* dest,
size_t len, enum encoding enc, uv_fs_cb after,
Func fn, Args... fn_args) {
CHECK_NOT_NULL(req_wrap);
req_wrap->Init(syscall, dest, len, enc);
int err = req_wrap->Dispatch(fn, fn_args..., after);
if (err < 0) {
uv_fs_t* uv_req = req_wrap->req();
uv_req->result = err;
uv_req->path = nullptr;
after(uv_req); // after may delete req_wrap if there is an error
req_wrap = nullptr;
} else {
req_wrap->SetReturnValue(args);
}
return req_wrap;
}
// Returns nullptr if the operation fails from the start.
template <typename Func, typename... Args>
inline FSReqBase* AsyncCall(Environment* env,
FSReqBase* req_wrap,
const v8::FunctionCallbackInfo<Value>& args,
const char* syscall, enum encoding enc,
uv_fs_cb after, Func fn, Args... fn_args) {
return AsyncDestCall(env, req_wrap, args,
syscall, nullptr, 0, enc,
after, fn, fn_args...);
}
// Template counterpart of SYNC_CALL, except that it only puts
// the error number and the syscall in the context instead of
// creating an error in the C++ land.
// ctx must be checked using value->IsObject() before being passed.
template <typename Func, typename... Args>
inline int SyncCall(Environment* env, v8::Local<v8::Value> ctx,
FSReqWrapSync* req_wrap, const char* syscall,
Func fn, Args... args) {
env->PrintSyncTrace();
int err = fn(env->event_loop(), &(req_wrap->req), args..., nullptr);
if (err < 0) {
v8::Local<Context> context = env->context();
v8::Local<Object> ctx_obj = ctx.As<v8::Object>();
v8::Isolate* isolate = env->isolate();
ctx_obj->Set(context,
env->errno_string(),
v8::Integer::New(isolate, err)).Check();
ctx_obj->Set(context,
env->syscall_string(),
OneByteString(isolate, syscall)).Check();
}
return err;
}
} // namespace fs
} // namespace node

View File

@@ -17,6 +17,7 @@ const expectedModules = new Set([
'Internal Binding contextify',
'Internal Binding credentials',
'Internal Binding fs',
'Internal Binding fs_dir',
'Internal Binding inspector',
'Internal Binding module_wrap',
'Internal Binding native_module',
@@ -42,6 +43,7 @@ const expectedModules = new Set([
'NativeModule internal/encoding',
'NativeModule internal/errors',
'NativeModule internal/fixed_queue',
'NativeModule internal/fs/dir',
'NativeModule internal/fs/utils',
'NativeModule internal/idna',
'NativeModule internal/linkedlist',

View File

@@ -0,0 +1,174 @@
'use strict';
const common = require('../common');
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const tmpdir = require('../common/tmpdir');
const testDir = tmpdir.path;
const files = ['empty', 'files', 'for', 'just', 'testing'];
// Make sure tmp directory is clean
tmpdir.refresh();
// Create the necessary files
files.forEach(function(filename) {
fs.closeSync(fs.openSync(path.join(testDir, filename), 'w'));
});
function assertDirent(dirent) {
assert(dirent instanceof fs.Dirent);
assert.strictEqual(dirent.isFile(), true);
assert.strictEqual(dirent.isDirectory(), false);
assert.strictEqual(dirent.isSocket(), false);
assert.strictEqual(dirent.isBlockDevice(), false);
assert.strictEqual(dirent.isCharacterDevice(), false);
assert.strictEqual(dirent.isFIFO(), false);
assert.strictEqual(dirent.isSymbolicLink(), false);
}
const dirclosedError = {
code: 'ERR_DIR_CLOSED'
};
// Check the opendir Sync version
{
const dir = fs.opendirSync(testDir);
const entries = files.map(() => {
const dirent = dir.readSync();
assertDirent(dirent);
return dirent.name;
});
assert.deepStrictEqual(files, entries.sort());
// dir.read should return null when no more entries exist
assert.strictEqual(dir.readSync(), null);
// check .path
assert.strictEqual(dir.path, testDir);
dir.closeSync();
assert.throws(() => dir.readSync(), dirclosedError);
assert.throws(() => dir.closeSync(), dirclosedError);
}
// Check the opendir async version
fs.opendir(testDir, common.mustCall(function(err, dir) {
assert.ifError(err);
dir.read(common.mustCall(function(err, dirent) {
assert.ifError(err);
// Order is operating / file system dependent
assert(files.includes(dirent.name), `'files' should include ${dirent}`);
assertDirent(dirent);
dir.close(common.mustCall(function(err) {
assert.ifError(err);
}));
}));
}));
// opendir() on file should throw ENOTDIR
assert.throws(function() {
fs.opendirSync(__filename);
}, /Error: ENOTDIR: not a directory/);
fs.opendir(__filename, common.mustCall(function(e) {
assert.strictEqual(e.code, 'ENOTDIR');
}));
[false, 1, [], {}, null, undefined].forEach((i) => {
common.expectsError(
() => fs.opendir(i, common.mustNotCall()),
{
code: 'ERR_INVALID_ARG_TYPE',
type: TypeError
}
);
common.expectsError(
() => fs.opendirSync(i),
{
code: 'ERR_INVALID_ARG_TYPE',
type: TypeError
}
);
});
// Promise-based tests
async function doPromiseTest() {
// Check the opendir Promise version
const dir = await fs.promises.opendir(testDir);
const entries = [];
let i = files.length;
while (i--) {
const dirent = await dir.read();
entries.push(dirent.name);
assertDirent(dirent);
}
assert.deepStrictEqual(files, entries.sort());
// dir.read should return null when no more entries exist
assert.strictEqual(await dir.read(), null);
await dir.close();
}
doPromiseTest().then(common.mustCall());
// Async iterator
async function doAsyncIterTest() {
const entries = [];
for await (const dirent of await fs.promises.opendir(testDir)) {
entries.push(dirent.name);
assertDirent(dirent);
}
assert.deepStrictEqual(files, entries.sort());
// Automatically closed during iterator
}
doAsyncIterTest().then(common.mustCall());
// Async iterators should do automatic cleanup
async function doAsyncIterBreakTest() {
const dir = await fs.promises.opendir(testDir);
for await (const dirent of dir) { // eslint-disable-line no-unused-vars
break;
}
await assert.rejects(async () => dir.read(), dirclosedError);
}
doAsyncIterBreakTest().then(common.mustCall());
async function doAsyncIterReturnTest() {
const dir = await fs.promises.opendir(testDir);
await (async function() {
for await (const dirent of dir) { // eslint-disable-line no-unused-vars
return;
}
})();
await assert.rejects(async () => dir.read(), dirclosedError);
}
doAsyncIterReturnTest().then(common.mustCall());
async function doAsyncIterThrowTest() {
const dir = await fs.promises.opendir(testDir);
try {
for await (const dirent of dir) { // eslint-disable-line no-unused-vars
throw new Error('oh no');
}
} catch (err) {
if (err.message !== 'oh no') {
throw err;
}
}
await assert.rejects(async () => dir.read(), dirclosedError);
}
doAsyncIterThrowTest().then(common.mustCall());

View File

@@ -306,3 +306,10 @@ if (process.features.inspector && common.isMainThread) {
{
v8.getHeapSnapshot().destroy();
}
// DIRHANDLE
{
const dirBinding = internalBinding('fs_dir');
const handle = dirBinding.opendir('./', 'utf8', undefined, {});
testInitialized(handle, 'DirHandle');
}

View File

@@ -72,6 +72,7 @@ const customTypesMap = {
'EventEmitter': 'events.html#events_class_eventemitter',
'FileHandle': 'fs.html#fs_class_filehandle',
'fs.Dir': 'fs.html#fs_class_fs_dir',
'fs.Dirent': 'fs.html#fs_class_fs_dirent',
'fs.FSWatcher': 'fs.html#fs_class_fs_fswatcher',
'fs.ReadStream': 'fs.html#fs_class_fs_readstream',