市场夺宝奇兵
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

303 lines
10 KiB

  1. # get-stream
  2. > Get a stream as a string, Buffer, ArrayBuffer or array
  3. ## Features
  4. - Works in any JavaScript environment ([Node.js](#nodejs-streams), [browsers](#browser-support), etc.).
  5. - Supports [text streams](#getstreamstream-options), [binary streams](#getstreamasbufferstream-options) and [object streams](#getstreamasarraystream-options).
  6. - Supports [async iterables](#async-iterables).
  7. - Can set a [maximum stream size](#maxbuffer).
  8. - Returns [partially read data](#errors) when the stream errors.
  9. - [Fast](#benchmarks).
  10. ## Install
  11. ```sh
  12. npm install get-stream
  13. ```
  14. ## Usage
  15. ### Node.js streams
  16. ```js
  17. import fs from 'node:fs';
  18. import getStream from 'get-stream';
  19. const stream = fs.createReadStream('unicorn.txt');
  20. console.log(await getStream(stream));
  21. /*
  22. ,,))))))));,
  23. __)))))))))))))),
  24. \|/ -\(((((''''((((((((.
  25. -*-==//////(('' . `)))))),
  26. /|\ ))| o ;-. '((((( ,(,
  27. ( `| / ) ;))))' ,_))^;(~
  28. | | | ,))((((_ _____------~~~-. %,;(;(>';'~
  29. o_); ; )))(((` ~---~ `:: \ %%~~)(v;(`('~
  30. ; ''''```` `: `:::|\,__,%% );`'; ~
  31. | _ ) / `:|`----' `-'
  32. ______/\/~ | / /
  33. /~;;.____/;;' / ___--,-( `;;;/
  34. / // _;______;'------~~~~~ /;;/\ /
  35. // | | / ; \;;,\
  36. (<_ | ; /',/-----' _>
  37. \_| ||_ //~;~~~~~~~~~
  38. `\_| (,~~
  39. \~\
  40. ~~
  41. */
  42. ```
  43. ### Web streams
  44. ```js
  45. import getStream from 'get-stream';
  46. const {body: readableStream} = await fetch('https://example.com');
  47. console.log(await getStream(readableStream));
  48. ```
  49. This works in any browser, even [the ones](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#browser_compatibility) not supporting `ReadableStream.values()` yet.
  50. ### Async iterables
  51. ```js
  52. import {opendir} from 'node:fs/promises';
  53. import {getStreamAsArray} from 'get-stream';
  54. const asyncIterable = await opendir(directory);
  55. console.log(await getStreamAsArray(asyncIterable));
  56. ```
  57. ## API
  58. The following methods read the stream's contents and return it as a promise.
  59. ### getStream(stream, options?)
  60. `stream`: [`stream.Readable`](https://nodejs.org/api/stream.html#class-streamreadable), [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream), or [`AsyncIterable<string | Buffer | ArrayBuffer | DataView | TypedArray>`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_async_iterator_and_async_iterable_protocols)\
  61. `options`: [`Options`](#options)
  62. Get the given `stream` as a string.
  63. ### getStreamAsBuffer(stream, options?)
  64. Get the given `stream` as a Node.js [`Buffer`](https://nodejs.org/api/buffer.html#class-buffer).
  65. ```js
  66. import {getStreamAsBuffer} from 'get-stream';
  67. const stream = fs.createReadStream('unicorn.png');
  68. console.log(await getStreamAsBuffer(stream));
  69. ```
  70. ### getStreamAsArrayBuffer(stream, options?)
  71. Get the given `stream` as an [`ArrayBuffer`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer).
  72. ```js
  73. import {getStreamAsArrayBuffer} from 'get-stream';
  74. const {body: readableStream} = await fetch('https://example.com');
  75. console.log(await getStreamAsArrayBuffer(readableStream));
  76. ```
  77. ### getStreamAsArray(stream, options?)
  78. Get the given `stream` as an array. Unlike [other methods](#api), this supports [streams of objects](https://nodejs.org/api/stream.html#object-mode).
  79. ```js
  80. import {getStreamAsArray} from 'get-stream';
  81. const {body: readableStream} = await fetch('https://example.com');
  82. console.log(await getStreamAsArray(readableStream));
  83. ```
  84. #### options
  85. Type: `object`
  86. ##### maxBuffer
  87. Type: `number`\
  88. Default: `Infinity`
  89. Maximum length of the stream. If exceeded, the promise will be rejected with a `MaxBufferError`.
  90. Depending on the [method](#api), the length is measured with [`string.length`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/length), [`buffer.length`](https://nodejs.org/api/buffer.html#buflength), [`arrayBuffer.byteLength`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer/byteLength) or [`array.length`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/length).
  91. ## Errors
  92. If the stream errors, the returned promise will be rejected with the `error`. Any contents already read from the stream will be set to `error.bufferedData`, which is a `string`, a `Buffer`, an `ArrayBuffer` or an array depending on the [method used](#api).
  93. ```js
  94. import getStream from 'get-stream';
  95. try {
  96. await getStream(streamThatErrorsAtTheEnd('unicorn'));
  97. } catch (error) {
  98. console.log(error.bufferedData);
  99. //=> 'unicorn'
  100. }
  101. ```
  102. ## Browser support
  103. For this module to work in browsers, a bundler must be used that either:
  104. - Supports the [`exports.browser`](https://nodejs.org/api/packages.html#community-conditions-definitions) field in `package.json`
  105. - Strips or ignores `node:*` imports
  106. Most bundlers (such as [Webpack](https://webpack.js.org/guides/package-exports/#target-environment)) support either of these.
  107. Additionally, browsers support [web streams](#web-streams) and [async iterables](#async-iterables), but not [Node.js streams](#nodejs-streams).
  108. ## Tips
  109. ### Alternatives
  110. If you do not need the [`maxBuffer`](#maxbuffer) option, [`error.bufferedData`](#errors), nor browser support, you can use the following methods instead of this package.
  111. #### [`streamConsumers.text()`](https://nodejs.org/api/webstreams.html#streamconsumerstextstream)
  112. ```js
  113. import fs from 'node:fs';
  114. import {text} from 'node:stream/consumers';
  115. const stream = fs.createReadStream('unicorn.txt', {encoding: 'utf8'});
  116. console.log(await text(stream))
  117. ```
  118. #### [`streamConsumers.buffer()`](https://nodejs.org/api/webstreams.html#streamconsumersbufferstream)
  119. ```js
  120. import {buffer} from 'node:stream/consumers';
  121. console.log(await buffer(stream))
  122. ```
  123. #### [`streamConsumers.arrayBuffer()`](https://nodejs.org/api/webstreams.html#streamconsumersarraybufferstream)
  124. ```js
  125. import {arrayBuffer} from 'node:stream/consumers';
  126. console.log(await arrayBuffer(stream))
  127. ```
  128. #### [`readable.toArray()`](https://nodejs.org/api/stream.html#readabletoarrayoptions)
  129. ```js
  130. console.log(await stream.toArray())
  131. ```
  132. #### [`Array.fromAsync()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fromAsync)
  133. If your [environment supports it](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fromAsync#browser_compatibility):
  134. ```js
  135. console.log(await Array.fromAsync(stream))
  136. ```
  137. ### Non-UTF-8 encoding
  138. When all of the following conditions apply:
  139. - [`getStream()`](#getstreamstream-options) is used (as opposed to [`getStreamAsBuffer()`](#getstreamasbufferstream-options) or [`getStreamAsArrayBuffer()`](#getstreamasarraybufferstream-options))
  140. - The stream is binary (not text)
  141. - The stream's encoding is not UTF-8 (for example, it is UTF-16, hexadecimal, or Base64)
  142. Then the stream must be decoded using a transform stream like [`TextDecoderStream`](https://developer.mozilla.org/en-US/docs/Web/API/TextDecoderStream) or [`b64`](https://github.com/hapijs/b64).
  143. ```js
  144. import getStream from 'get-stream';
  145. const textDecoderStream = new TextDecoderStream('utf-16le');
  146. const {body: readableStream} = await fetch('https://example.com');
  147. console.log(await getStream(readableStream.pipeThrough(textDecoderStream)));
  148. ```
  149. ### Blobs
  150. [`getStreamAsArrayBuffer()`](#getstreamasarraybufferstream-options) can be used to create [Blobs](https://developer.mozilla.org/en-US/docs/Web/API/Blob).
  151. ```js
  152. import {getStreamAsArrayBuffer} from 'get-stream';
  153. const stream = fs.createReadStream('unicorn.txt');
  154. console.log(new Blob([await getStreamAsArrayBuffer(stream)]));
  155. ```
  156. ### JSON streaming
  157. [`getStreamAsArray()`](#getstreamasarraystream-options) can be combined with JSON streaming utilities to parse JSON incrementally.
  158. ```js
  159. import fs from 'node:fs';
  160. import {compose as composeStreams} from 'node:stream';
  161. import {getStreamAsArray} from 'get-stream';
  162. import streamJson from 'stream-json';
  163. import streamJsonArray from 'stream-json/streamers/StreamArray.js';
  164. const stream = fs.createReadStream('big-array-of-objects.json');
  165. console.log(await getStreamAsArray(
  166. composeStreams(stream, streamJson.parser(), streamJsonArray.streamArray()),
  167. ));
  168. ```
  169. ## Benchmarks
  170. ### Node.js stream (100 MB, binary)
  171. - `getStream()`: 142ms
  172. - `text()`: 139ms
  173. - `getStreamAsBuffer()`: 106ms
  174. - `buffer()`: 83ms
  175. - `getStreamAsArrayBuffer()`: 105ms
  176. - `arrayBuffer()`: 81ms
  177. - `getStreamAsArray()`: 24ms
  178. - `stream.toArray()`: 21ms
  179. ### Node.js stream (100 MB, text)
  180. - `getStream()`: 90ms
  181. - `text()`: 89ms
  182. - `getStreamAsBuffer()`: 127ms
  183. - `buffer()`: 192ms
  184. - `getStreamAsArrayBuffer()`: 129ms
  185. - `arrayBuffer()`: 195ms
  186. - `getStreamAsArray()`: 89ms
  187. - `stream.toArray()`: 90ms
  188. ### Web ReadableStream (100 MB, binary)
  189. - `getStream()`: 223ms
  190. - `text()`: 221ms
  191. - `getStreamAsBuffer()`: 182ms
  192. - `buffer()`: 153ms
  193. - `getStreamAsArrayBuffer()`: 171ms
  194. - `arrayBuffer()`: 155ms
  195. - `getStreamAsArray()`: 83ms
  196. ### Web ReadableStream (100 MB, text)
  197. - `getStream()`: 141ms
  198. - `text()`: 139ms
  199. - `getStreamAsBuffer()`: 91ms
  200. - `buffer()`: 80ms
  201. - `getStreamAsArrayBuffer()`: 89ms
  202. - `arrayBuffer()`: 81ms
  203. - `getStreamAsArray()`: 21ms
  204. [Benchmarks' source file](benchmarks/index.js).
  205. ## FAQ
  206. ### How is this different from [`concat-stream`](https://github.com/maxogden/concat-stream)?
  207. This module accepts a stream instead of being one and returns a promise instead of using a callback. The API is simpler and it only supports returning a string, `Buffer`, an `ArrayBuffer` or an array. It doesn't have a fragile type inference. You explicitly choose what you want. And it doesn't depend on the huge `readable-stream` package.
  208. ## Related
  209. - [get-stdin](https://github.com/sindresorhus/get-stdin) - Get stdin as a string or buffer
  210. - [into-stream](https://github.com/sindresorhus/into-stream) - The opposite of this package