This patch implements proxy support for HTTP and HTTPS clients and agents in the `http` and `https` built-ins`. When NODE_USE_ENV_PROXY is set to 1, the default global agent would parse the HTTP_PROXY/http_proxy, HTTPS_PROXY/https_proxy, NO_PROXY/no_proxy settings from the environment variables, and proxy the requests sent through the built-in http/https client accordingly. To support this, `http.Agent` and `https.Agent` now accept a few new options: - `proxyEnv`: when it's an object, the agent would read and parse the HTTP_PROXY/http_proxy, HTTPS_PROXY/https_proxy, NO_PROXY/no_proxy properties from it, and apply them based on the protocol it uses to send requests. This option allows custom agents to reuse built-in proxy support by composing options. Global agents set this to `process.env` when NODE_USE_ENV_PROXY is 1. - `defaultPort` and `protocol`: these allow setting of the default port and protocol of the agents. We also need these when configuring proxy settings and deciding whether a request should be proxied. Implementation-wise, this adds a `ProxyConfig` internal class to handle parsing and application of proxy configurations. The configuration is parsed during agent construction. When requests are made, the `createConnection()` methods on the agents would check whether the request should be proxied. If yes, they either connect to the proxy server (in the case of HTTP reqeusts) or establish a tunnel (in the case of HTTPS requests) through either a TCP socket (if the proxy uses HTTP) or a TLS socket (if the proxy uses HTTPS). When proxying HTTPS requests through a tunnel, the connection listener is invoked after the tunnel is established. Tunnel establishment uses the timeout of the request options, if there is one. Otherwise it uses the timeout of the agent. If an error is encountered during tunnel establishment, an ERR_PROXY_TUNNEL would be emitted on the returned socket. If the proxy server sends a errored status code, the error would contain an `statusCode` property. If the error is caused by timeout, the error would contain a `proxyTunnelTimeout` property. This implementation honors the built-in socket pool and socket limits. Pooled sockets are still keyed by request endpoints, they are just connected to the proxy server instead, and the persistence of the connection can be maintained as long as the proxy server respects connection/proxy-connection or persist by default (HTTP/1.1) PR-URL: https://github.com/nodejs/node/pull/58980 Refs: https://github.com/nodejs/node/issues/57872 Refs: https://github.com/nodejs/node/issues/8381 Refs: https://github.com/nodejs/node/issues/15620 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
21 KiB
HTTPS
Stability: 2 - Stable
HTTPS is the HTTP protocol over TLS/SSL. In Node.js this is implemented as a separate module.
Determining if crypto support is unavailable
It is possible for Node.js to be built without including support for the
node:crypto module. In such cases, attempting to import from https or
calling require('node:https') will result in an error being thrown.
When using CommonJS, the error thrown can be caught using try/catch:
let https;
try {
https = require('node:https');
} catch (err) {
console.error('https support is disabled!');
}
When using the lexical ESM import keyword, the error can only be
caught if a handler for process.on('uncaughtException') is registered
before any attempt to load the module is made (using, for instance,
a preload module).
When using ESM, if there is a chance that the code may be run on a build
of Node.js where crypto support is not enabled, consider using the
import() function instead of the lexical import keyword:
let https;
try {
https = await import('node:https');
} catch (err) {
console.error('https support is disabled!');
}
Class: https.Agent
An Agent object for HTTPS similar to http.Agent. See
https.request() for more information.
new Agent([options])
options{Object} Set of configurable options to set on the agent. Can have the same fields as forhttp.Agent(options), and-
maxCachedSessions{number} maximum number of TLS cached sessions. Use0to disable TLS session caching. Default:100. -
servername{string} the value of Server Name Indication extension to be sent to the server. Use empty string''to disable sending the extension. Default: host name of the target server, unless the target server is specified using an IP address, in which case the default is''(no extension).See
Session Resumptionfor information about TLS session reuse.
-
Event: 'keylog'
line{Buffer} Line of ASCII text, in NSSSSLKEYLOGFILEformat.tlsSocket{tls.TLSSocket} Thetls.TLSSocketinstance on which it was generated.
The keylog event is emitted when key material is generated or received by a
connection managed by this agent (typically before handshake has completed, but
not necessarily). This keying material can be stored for debugging, as it
allows captured TLS traffic to be decrypted. It may be emitted multiple times
for each socket.
A typical use case is to append received lines to a common text file, which is later used by software (such as Wireshark) to decrypt the traffic:
// ...
https.globalAgent.on('keylog', (line, tlsSocket) => {
fs.appendFileSync('/tmp/ssl-keys.log', line, { mode: 0o600 });
});
Class: https.Server
- Extends: {tls.Server}
See http.Server for more information.
server.close([callback])
callback{Function}- Returns: {https.Server}
See server.close() in the node:http module.
server[Symbol.asyncDispose]()
Calls server.close() and returns a promise that
fulfills when the server has closed.
server.closeAllConnections()
See server.closeAllConnections() in the node:http module.
server.closeIdleConnections()
See server.closeIdleConnections() in the node:http module.
server.headersTimeout
- Type: {number} Default:
60000
See server.headersTimeout in the node:http module.
server.listen()
Starts the HTTPS server listening for encrypted connections.
This method is identical to server.listen() from net.Server.
server.maxHeadersCount
- Type: {number} Default:
2000
See server.maxHeadersCount in the node:http module.
server.requestTimeout
- Type: {number} Default:
300000
See server.requestTimeout in the node:http module.
server.setTimeout([msecs][, callback])
msecs{number} Default:120000(2 minutes)callback{Function}- Returns: {https.Server}
See server.setTimeout() in the node:http module.
server.timeout
- Type: {number} Default: 0 (no timeout)
See server.timeout in the node:http module.
server.keepAliveTimeout
- Type: {number} Default:
5000(5 seconds)
See server.keepAliveTimeout in the node:http module.
https.createServer([options][, requestListener])
options{Object} Acceptsoptionsfromtls.createServer(),tls.createSecureContext()andhttp.createServer().requestListener{Function} A listener to be added to the'request'event.- Returns: {https.Server}
// curl -k https://localhost:8000/
import { createServer } from 'node:https';
import { readFileSync } from 'node:fs';
const options = {
key: readFileSync('private-key.pem'),
cert: readFileSync('certificate.pem'),
};
createServer(options, (req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
// curl -k https://localhost:8000/
const https = require('node:https');
const fs = require('node:fs');
const options = {
key: fs.readFileSync('private-key.pem'),
cert: fs.readFileSync('certificate.pem'),
};
https.createServer(options, (req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
Or
import { createServer } from 'node:https';
import { readFileSync } from 'node:fs';
const options = {
pfx: readFileSync('test_cert.pfx'),
passphrase: 'sample',
};
createServer(options, (req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
const https = require('node:https');
const fs = require('node:fs');
const options = {
pfx: fs.readFileSync('test_cert.pfx'),
passphrase: 'sample',
};
https.createServer(options, (req, res) => {
res.writeHead(200);
res.end('hello world\n');
}).listen(8000);
To generate the certificate and key for this example, run:
openssl req -x509 -newkey rsa:2048 -nodes -sha256 -subj '/CN=localhost' \
-keyout private-key.pem -out certificate.pem
Then, to generate the pfx certificate for this example, run:
openssl pkcs12 -certpbe AES-256-CBC -export -out test_cert.pfx \
-inkey private-key.pem -in certificate.pem -passout pass:sample
https.get(options[, callback])
https.get(url[, options][, callback])
url{string | URL}options{Object | string | URL} Accepts the sameoptionsashttps.request(), with the method set to GET by default.callback{Function}- Returns: {http.ClientRequest}
Like http.get() but for HTTPS.
options can be an object, a string, or a URL object. If options is a
string, it is automatically parsed with new URL(). If it is a URL
object, it will be automatically converted to an ordinary options object.
import { get } from 'node:https';
import process from 'node:process';
get('https://encrypted.google.com/', (res) => {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
res.on('data', (d) => {
process.stdout.write(d);
});
}).on('error', (e) => {
console.error(e);
});
const https = require('node:https');
https.get('https://encrypted.google.com/', (res) => {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
res.on('data', (d) => {
process.stdout.write(d);
});
}).on('error', (e) => {
console.error(e);
});
https.globalAgent
Global instance of https.Agent for all HTTPS client requests. Diverges
from a default https.Agent configuration by having keepAlive enabled and
a timeout of 5 seconds.
https.request(options[, callback])
https.request(url[, options][, callback])
url{string | URL}options{Object | string | URL} Accepts alloptionsfromhttp.request(), with some differences in default values:protocolDefault:'https:'portDefault:443agentDefault:https.globalAgent
callback{Function}- Returns: {http.ClientRequest}
Makes a request to a secure web server.
The following additional options from tls.connect() are also accepted:
ca, cert, ciphers, clientCertEngine (deprecated), crl, dhparam, ecdhCurve,
honorCipherOrder, key, passphrase, pfx, rejectUnauthorized,
secureOptions, secureProtocol, servername, sessionIdContext,
highWaterMark.
options can be an object, a string, or a URL object. If options is a
string, it is automatically parsed with new URL(). If it is a URL
object, it will be automatically converted to an ordinary options object.
https.request() returns an instance of the http.ClientRequest
class. The ClientRequest instance is a writable stream. If one needs to
upload a file with a POST request, then write to the ClientRequest object.
import { request } from 'node:https';
import process from 'node:process';
const options = {
hostname: 'encrypted.google.com',
port: 443,
path: '/',
method: 'GET',
};
const req = request(options, (res) => {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
res.on('data', (d) => {
process.stdout.write(d);
});
});
req.on('error', (e) => {
console.error(e);
});
req.end();
const https = require('node:https');
const options = {
hostname: 'encrypted.google.com',
port: 443,
path: '/',
method: 'GET',
};
const req = https.request(options, (res) => {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
res.on('data', (d) => {
process.stdout.write(d);
});
});
req.on('error', (e) => {
console.error(e);
});
req.end();
Example using options from tls.connect():
const options = {
hostname: 'encrypted.google.com',
port: 443,
path: '/',
method: 'GET',
key: fs.readFileSync('private-key.pem'),
cert: fs.readFileSync('certificate.pem'),
};
options.agent = new https.Agent(options);
const req = https.request(options, (res) => {
// ...
});
Alternatively, opt out of connection pooling by not using an Agent.
const options = {
hostname: 'encrypted.google.com',
port: 443,
path: '/',
method: 'GET',
key: fs.readFileSync('private-key.pem'),
cert: fs.readFileSync('certificate.pem'),
agent: false,
};
const req = https.request(options, (res) => {
// ...
});
Example using a URL as options:
const options = new URL('https://abc:xyz@example.com');
const req = https.request(options, (res) => {
// ...
});
Example pinning on certificate fingerprint, or the public key (similar to
pin-sha256):
import { checkServerIdentity } from 'node:tls';
import { Agent, request } from 'node:https';
import { createHash } from 'node:crypto';
function sha256(s) {
return createHash('sha256').update(s).digest('base64');
}
const options = {
hostname: 'github.com',
port: 443,
path: '/',
method: 'GET',
checkServerIdentity: function(host, cert) {
// Make sure the certificate is issued to the host we are connected to
const err = checkServerIdentity(host, cert);
if (err) {
return err;
}
// Pin the public key, similar to HPKP pin-sha256 pinning
const pubkey256 = 'SIXvRyDmBJSgatgTQRGbInBaAK+hZOQ18UmrSwnDlK8=';
if (sha256(cert.pubkey) !== pubkey256) {
const msg = 'Certificate verification error: ' +
`The public key of '${cert.subject.CN}' ` +
'does not match our pinned fingerprint';
return new Error(msg);
}
// Pin the exact certificate, rather than the pub key
const cert256 = 'FD:6E:9B:0E:F3:98:BC:D9:04:C3:B2:EC:16:7A:7B:' +
'0F:DA:72:01:C9:03:C5:3A:6A:6A:E5:D0:41:43:63:EF:65';
if (cert.fingerprint256 !== cert256) {
const msg = 'Certificate verification error: ' +
`The certificate of '${cert.subject.CN}' ` +
'does not match our pinned fingerprint';
return new Error(msg);
}
// This loop is informational only.
// Print the certificate and public key fingerprints of all certs in the
// chain. Its common to pin the public key of the issuer on the public
// internet, while pinning the public key of the service in sensitive
// environments.
let lastprint256;
do {
console.log('Subject Common Name:', cert.subject.CN);
console.log(' Certificate SHA256 fingerprint:', cert.fingerprint256);
const hash = createHash('sha256');
console.log(' Public key ping-sha256:', sha256(cert.pubkey));
lastprint256 = cert.fingerprint256;
cert = cert.issuerCertificate;
} while (cert.fingerprint256 !== lastprint256);
},
};
options.agent = new Agent(options);
const req = request(options, (res) => {
console.log('All OK. Server matched our pinned cert or public key');
console.log('statusCode:', res.statusCode);
res.on('data', (d) => {});
});
req.on('error', (e) => {
console.error(e.message);
});
req.end();
const tls = require('node:tls');
const https = require('node:https');
const crypto = require('node:crypto');
function sha256(s) {
return crypto.createHash('sha256').update(s).digest('base64');
}
const options = {
hostname: 'github.com',
port: 443,
path: '/',
method: 'GET',
checkServerIdentity: function(host, cert) {
// Make sure the certificate is issued to the host we are connected to
const err = tls.checkServerIdentity(host, cert);
if (err) {
return err;
}
// Pin the public key, similar to HPKP pin-sha256 pinning
const pubkey256 = 'SIXvRyDmBJSgatgTQRGbInBaAK+hZOQ18UmrSwnDlK8=';
if (sha256(cert.pubkey) !== pubkey256) {
const msg = 'Certificate verification error: ' +
`The public key of '${cert.subject.CN}' ` +
'does not match our pinned fingerprint';
return new Error(msg);
}
// Pin the exact certificate, rather than the pub key
const cert256 = 'FD:6E:9B:0E:F3:98:BC:D9:04:C3:B2:EC:16:7A:7B:' +
'0F:DA:72:01:C9:03:C5:3A:6A:6A:E5:D0:41:43:63:EF:65';
if (cert.fingerprint256 !== cert256) {
const msg = 'Certificate verification error: ' +
`The certificate of '${cert.subject.CN}' ` +
'does not match our pinned fingerprint';
return new Error(msg);
}
// This loop is informational only.
// Print the certificate and public key fingerprints of all certs in the
// chain. Its common to pin the public key of the issuer on the public
// internet, while pinning the public key of the service in sensitive
// environments.
do {
console.log('Subject Common Name:', cert.subject.CN);
console.log(' Certificate SHA256 fingerprint:', cert.fingerprint256);
hash = crypto.createHash('sha256');
console.log(' Public key ping-sha256:', sha256(cert.pubkey));
lastprint256 = cert.fingerprint256;
cert = cert.issuerCertificate;
} while (cert.fingerprint256 !== lastprint256);
},
};
options.agent = new https.Agent(options);
const req = https.request(options, (res) => {
console.log('All OK. Server matched our pinned cert or public key');
console.log('statusCode:', res.statusCode);
res.on('data', (d) => {});
});
req.on('error', (e) => {
console.error(e.message);
});
req.end();
Outputs for example:
Subject Common Name: github.com
Certificate SHA256 fingerprint: FD:6E:9B:0E:F3:98:BC:D9:04:C3:B2:EC:16:7A:7B:0F:DA:72:01:C9:03:C5:3A:6A:6A:E5:D0:41:43:63:EF:65
Public key ping-sha256: SIXvRyDmBJSgatgTQRGbInBaAK+hZOQ18UmrSwnDlK8=
Subject Common Name: Sectigo ECC Domain Validation Secure Server CA
Certificate SHA256 fingerprint: 61:E9:73:75:E9:F6:DA:98:2F:F5:C1:9E:2F:94:E6:6C:4E:35:B6:83:7C:E3:B9:14:D2:24:5C:7F:5F:65:82:5F
Public key ping-sha256: Eep0p/AsSa9lFUH6KT2UY+9s1Z8v7voAPkQ4fGknZ2g=
Subject Common Name: USERTrust ECC Certification Authority
Certificate SHA256 fingerprint: A6:CF:64:DB:B4:C8:D5:FD:19:CE:48:89:60:68:DB:03:B5:33:A8:D1:33:6C:62:56:A8:7D:00:CB:B3:DE:F3:EA
Public key ping-sha256: UJM2FOhG9aTNY0Pg4hgqjNzZ/lQBiMGRxPD5Y2/e0bw=
Subject Common Name: AAA Certificate Services
Certificate SHA256 fingerprint: D7:A7:A0:FB:5D:7E:27:31:D7:71:E9:48:4E:BC:DE:F7:1D:5F:0C:3E:0A:29:48:78:2B:C8:3E:E0:EA:69:9E:F4
Public key ping-sha256: vRU+17BDT2iGsXvOi76E7TQMcTLXAqj0+jGPdW7L1vM=
All OK. Server matched our pinned cert or public key
statusCode: 200