# Object-Scan [![Build Status](https://circleci.com/gh/blackflux/object-scan.png?style=shield)](https://circleci.com/gh/blackflux/object-scan) [![Test Coverage](https://img.shields.io/coveralls/blackflux/object-scan/master.svg)](https://coveralls.io/github/blackflux/object-scan?branch=master) [![Dependabot Status](https://api.dependabot.com/badges/status?host=github&repo=blackflux/object-scan)](https://dependabot.com) [![Dependencies](https://david-dm.org/blackflux/object-scan/status.svg)](https://david-dm.org/blackflux/object-scan) [![NPM](https://img.shields.io/npm/v/object-scan.svg)](https://www.npmjs.com/package/object-scan) [![Downloads](https://img.shields.io/npm/dt/object-scan.svg)](https://www.npmjs.com/package/object-scan) [![Semantic-Release](https://github.com/blackflux/js-gardener/blob/master/assets/icons/semver.svg)](https://github.com/semantic-release/semantic-release) [![Gardener](https://github.com/blackflux/js-gardener/blob/master/assets/badge.svg)](https://github.com/blackflux/js-gardener) Traverse object hierarchies using matching and callbacks. ## Install Install with [npm](https://www.npmjs.com/): $ npm install --save object-scan ## Usage ```js const objectScan = require('object-scan'); const haystack = { a: { b: { c: 'd' }, e: { f: 'g' } } }; objectScan(['a.*.f'], { joined: true })(haystack); // => [ 'a.e.f' ] ``` ## Features - Input traversed exactly once during search - Dependency free, small in size and very performant - Separate Object and Array matching - Wildcard and Regex matching - Arbitrary depth matching - Or-clause Syntax - Exclusion Matching - Full support for escaping - Traversal in "delete-safe" order - Recursion free implementation - Search syntax validated - Lots of tests and examples ## Matching A needle expression specifies one or more paths to an element (or a set of elements) in a JSON structure. Paths use the dot notation: ```txt store.book[0].title ``` ### Array Rectangular brackets for array path matching. _Examples_:
['[2]'] (exact in array) ```js const haystack = [0, 1, 2, 3, 4]; objectScan(['[2]'], { joined: true })(haystack); // => [ '[2]' ] ```
['[1]'] (no match in object) ```js const haystack = { 0: 'a', 1: 'b', 2: 'c' }; objectScan(['[1]'], { joined: true })(haystack); // => [] ```
### Object Property name for object property matching. _Examples_:
['foo'] (exact in object) ```js const haystack = { foo: 0, bar: 1 }; objectScan(['foo'], { joined: true })(haystack); // => [ 'foo' ] ```
['1'] (no match in array) ```js const haystack = [0, 1, 2, 3, 4]; objectScan(['1'], { joined: true })(haystack); // => [] ```
### Wildcard The following characters have special meaning when not escaped: - `*`: Match zero or more character - `+`: Match one or more character - `?`: Match exactly one character - `\`: Escape the subsequent character Wildcards can be used with Array and Object selector. _Examples_:
['*'] (top level) ```js const haystack = { a: { b: 0, c: 1 }, d: 2 }; objectScan(['*'], { joined: true })(haystack); // => [ 'd', 'a' ] ```
['[?5]'] (two digit ending in five) ```js const haystack = [...Array(30).keys()]; objectScan(['[?5]'], { joined: true })(haystack); // => [ '[25]', '[15]' ] ```
['a.+.c'] (nested) ```js const haystack = { a: { b: { c: 0 }, d: { f: 0 } } }; objectScan(['a.+.c'], { joined: true })(haystack); // => [ 'a.b.c' ] ```
['a.\\+.c'] (escaped) ```js const haystack = { a: { b: { c: 0 }, '+': { c: 0 } } }; objectScan(['a.\\+.c'], { joined: true })(haystack); // => [ 'a.\\+.c' ] ```
### Regex Regex are defined by using parentheses. Can be used with Array and Object selector. _Examples_:
['(^foo)'] (starting with `foo`) ```js const haystack = { foo: 0, foobar: 1, bar: 2 }; objectScan(['(^foo)'], { joined: true })(haystack); // => [ 'foobar', 'foo' ] ```
['[(5)]'] (containing `5`) ```js const haystack = [...Array(20).keys()]; objectScan(['[(5)]'], { joined: true })(haystack); // => [ '[15]', '[5]' ] ```
['[(^[01]$)]'] (`[0]` and `[1]`) ```js const haystack = ['a', 'b', 'c', 'd']; objectScan(['[(^[01]$)]'], { joined: true })(haystack); // => [ '[1]', '[0]' ] ```
['[(^[^01]$)]'] (other than `[0]` and `[1]`) ```js const haystack = ['a', 'b', 'c', 'd']; objectScan(['[(^[^01]$)]'], { joined: true })(haystack); // => [ '[3]', '[2]' ] ```
['[*]', '[!(^[01]$)]'] (match all and exclude `[0]` and `[1]`) ```js const haystack = ['a', 'b', 'c', 'd']; objectScan(['[*]', '[!(^[01]$)]'], { joined: true })(haystack); // => [ '[3]', '[2]' ] ```
### Arbitrary Depth There are two types of arbitrary depth matching: - `**`: Matches zero or more nestings - `++`: Matches one or more nestings Recursions can be combined with a regex by appending the regex. _Examples_:
['a.**'] (zero or more nestings under `a`) ```js const haystack = { a: { b: 0, c: 0 } }; objectScan(['a.**'], { joined: true })(haystack); // => [ 'a.c', 'a.b', 'a' ] ```
['a.++'] (one or more nestings under `a`) ```js const haystack = { a: { b: 0, c: 0 } }; objectScan(['a.++'], { joined: true })(haystack); // => [ 'a.c', 'a.b' ] ```
['**(1)'] (all containing `1` at every level) ```js const haystack = { 1: { 1: ['c', 'd'] }, 510: 'e', foo: { 1: 'f' } }; objectScan(['**(1)'], { joined: true })(haystack); // => [ '510', '1.1[1]', '1.1', '1' ] ```
### Or Clause Or Clauses are defined by using curley brackets. Can be used with Array and Object selector. _Examples_:
['[{0,1}]'] (`[0]` and `[1]`) ```js const haystack = ['a', 'b', 'c', 'd']; objectScan(['[{0,1}]'], { joined: true })(haystack); // => [ '[1]', '[0]' ] ```
['{a,d}.{b,f}'] (`a.b`, `a.f`, `d.b` and `d.f`) ```js const haystack = { a: { b: 0, c: 1 }, d: { e: 2, f: 3 } }; objectScan(['{a,d}.{b,f}'], { joined: true })(haystack); // => [ 'd.f', 'a.b' ] ```
### Exclusion To exclude a path, use exclamation mark. _Examples_:
['{a,b},!a'] (only `b`) ```js const haystack = { a: 0, b: 1 }; objectScan(['{a,b},!a'], { joined: true, strict: false })(haystack); // => [ 'b' ] ```
['**,!**.a'] (all except ending in `a`) ```js const haystack = { a: 0, b: { a: 1, c: 2 } }; objectScan(['**,!**.a'], { joined: true })(haystack); // => [ 'b.c', 'b' ] ```
### Escaping The following characters are considered special and need to be escaped using `\`, if they should be matched in a key:
`[`, `]`, `{`, `}`, `(`, `)`, `,`, `.`, `!`, `?`, `*`, `+` and `\`. _Examples:_
['\\[1\\]'] (special object key) ```js const haystack = { '[1]': 0 }; objectScan(['\\[1\\]'], { joined: true })(haystack); // => [ '\\[1\\]' ] ```
## Options Signature of all callbacks is Fn({ key, value, ... }) where: - `key`: key that callback is invoked for (respects `joined` option). - `value`: value for key. - `entry`: entry consisting of [`key`, `value`]. - `property`: current parent property. - `parent`: current parent. - `parents`: array of form `[parent, grandparent, ...]`. - `isMatch`: true iff last targeting needle exists and is non-excluding. - `matchedBy`: all non-excluding needles targeting key. - `excludedBy`: all excluding needles targeting key. - `traversedBy`: all needles involved in traversing key. - `isCircular`: true iff `value` contained in `parents` - `isLeaf`: true iff `value` can not be traversed - `getKey`: function that returns `key` - `getValue`: function that returns `value` - `getEntry`: function that returns `entry` - `getProperty`: function that returns `property` - `getParent`: function that returns `parent` - `getParents`: function that returns `parents` - `getIsMatch`: function that returns `isMatch` - `getMatchedBy`: function that returns `matchedBy` - `getExcludedBy`: function that returns `excludedBy` - `getTraversedBy`: function that returns `traversedBy` - `getIsCircular`: function that returns `isCircular` - `getIsLeaf`: function that returns `isLeaf` - `context`: as passed into the search Notes on Performance: - Arguments backed by getters use [Functions Getter](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/get) and should be accessed via [destructuring](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment#Unpacking_fields_from_objects_passed_as_function_parameter) to prevent redundant computation. - Getters should be used to improve performance for conditional access. E.g. `if (isMatch) { getParents() ... }`. - For performance reasons, the same object is passed to all callbacks. #### filterFn Type: `function`
Default: `undefined` When defined, this callback is invoked for every match. If `false` is returned, the current key is excluded from the result. The return value of this callback has no effect when a search context is provided. Can be used to do processing as matching keys are traversed. Invoked in same order as matches would appear in result. This method is conceptually similar to [Array.filter()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter). _Examples_:
['**'] (filter function) ```js const haystack = { a: 0, b: 'bar' }; objectScan(['**'], { joined: true, filterFn: ({ value }) => typeof value === 'string' })(haystack); // => [ 'b' ] ```
#### breakFn Type: `function`
Default: `undefined` When defined, this callback is invoked for every key that is traversed by the search. If `true` is returned, all keys nested under the current key are skipped in the search and from the final result. Note that `breakFn` is invoked before the corresponding `filterFn` might be invoked. _Examples_:
['**'] (break function) ```js const haystack = { a: { b: { c: 0 } } }; objectScan(['**'], { joined: true, breakFn: ({ key }) => key === 'a.b' })(haystack); // => [ 'a.b', 'a' ] ```
#### compareFn Type: `function`
Default: `undefined` When defined, this function is used as a comparator to determine the traversal order of any `object` keys. This works together with the `reverse` option. _Examples_:
['**'] (simple sort) ```js const haystack = { a: 0, c: 1, b: 2 }; objectScan(['**'], { joined: true, compareFn: (k1, k2) => k1.localeCompare(k2), reverse: false })(haystack); // => [ 'a', 'b', 'c' ] ```
#### reverse Type: `boolean`
Default: `true` When set to `true`, the scan is performed in reverse order. This means `breakFn` is executed in _reverse post-order_ and `filterFn` in _reverse pre-order_. Otherwise `breakFn` is executed in _pre-order_ and `filterFn` in _post-order_. When `reverse` is `true` the scan is _delete-safe_. I.e. `property` can be deleted / spliced from `parent` object / array in `filterFn`. _Examples_:
['**'] (breakFn, reverse true) ```js const haystack = { f: { b: { a: {}, d: { c: {}, e: {} } }, g: { i: { h: {} } } } }; objectScan(['**'], { breakFn: ({ isMatch, property, context }) => { if (isMatch) { context.push(property); } }, reverse: true })(haystack, []); // => [ 'f', 'g', 'i', 'h', 'b', 'd', 'e', 'c', 'a' ] ```
['**'] (filterFn, reverse true) ```js const haystack = { f: { b: { a: {}, d: { c: {}, e: {} } }, g: { i: { h: {} } } } }; objectScan(['**'], { filterFn: ({ property, context }) => { context.push(property); }, reverse: true })(haystack, []); // => [ 'h', 'i', 'g', 'e', 'c', 'd', 'a', 'b', 'f' ] ```
['**'] (breakFn, reverse false) ```js const haystack = { f: { b: { a: {}, d: { c: {}, e: {} } }, g: { i: { h: {} } } } }; objectScan(['**'], { breakFn: ({ isMatch, property, context }) => { if (isMatch) { context.push(property); } }, reverse: false })(haystack, []); // => [ 'f', 'b', 'a', 'd', 'c', 'e', 'g', 'i', 'h' ] ```
['**'] (filterFn, reverse false) ```js const haystack = { f: { b: { a: {}, d: { c: {}, e: {} } }, g: { i: { h: {} } } } }; objectScan(['**'], { filterFn: ({ property, context }) => { context.push(property); }, reverse: false })(haystack, []); // => [ 'a', 'c', 'e', 'd', 'b', 'h', 'i', 'g', 'f' ] ```
#### abort Type: `boolean`
Default: `false` When set to `true` the scan immediately returns after the first match. _Examples_:
['a', 'b'] (only return first property) ```js const haystack = { a: 0, b: 1 }; objectScan(['a', 'b'], { rtn: 'property', abort: true })(haystack); // => 'b' ```
['[0]', '[1]'] (abort changes count) ```js const haystack = ['a', 'b']; objectScan(['[0]', '[1]'], { rtn: 'count', abort: true })(haystack); // => 1 ```
#### rtn Type: `string`
Default: _dynamic_ Defaults to `key` when search context is _undefined_ and to `context` otherwise. Can be explicitly set as: - `context`: search context is returned - `key`: matched keys are returned - `value`: matched values are returned - `entry`: matched entries are returned - `property`: matched properties are returned - `parent`: matched parent are returned - `parents`: matched parents are returned - `bool`: returns _true_ iff a match is found - `count`: returns the match count When **abort** is set to `true` and the result would be a list, the first match or _undefined_ is returned. _Examples_:
['[*]'] (return values) ```js const haystack = ['a', 'b', 'c']; objectScan(['[*]'], { rtn: 'value' })(haystack); // => [ 'c', 'b', 'a' ] ```
['foo[*]'] (return entries) ```js const haystack = { foo: ['bar'] }; objectScan(['foo[*]'], { rtn: 'entry' })(haystack); // => [ [ [ 'foo', 0 ], 'bar' ] ] ```
['a.b.c', 'a'] (return properties) ```js const haystack = { a: { b: { c: 0 } } }; objectScan(['a.b.c', 'a'], { rtn: 'property' })(haystack); // => [ 'c', 'a' ] ```
['a.b', 'a.c'] (checks for any match, full scan) ```js const haystack = { a: { b: 0, c: 1 } }; objectScan(['a.b', 'a.c'], { rtn: 'bool' })(haystack); // => true ```
['**'] (return not provided context) ```js const haystack = { a: 0 }; objectScan(['**'], { rtn: 'context' })(haystack); // => undefined ```
['a.b.{c,d}'] (return keys with context passed) ```js const haystack = { a: { b: { c: 0, d: 1 } } }; objectScan(['a.b.{c,d}'], { rtn: 'key' })(haystack, []); // => [ [ 'a', 'b', 'd' ], [ 'a', 'b', 'c' ] ] ```
#### joined Type: `boolean`
Default: `false` Keys are returned as a string when set to `true` instead of as a list. Setting this option to `true` will negatively impact performance. Note that [_.get](https://lodash.com/docs/#get) and [_.set](https://lodash.com/docs/#set) fully support lists. _Examples_:
['[*]', '[*].foo'] (joined) ```js const haystack = [0, 1, { foo: 'bar' }]; objectScan(['[*]', '[*].foo'], { joined: true })(haystack); // => [ '[2].foo', '[2]', '[1]', '[0]' ] ```
['[*]', '[*].foo'] (not joined) ```js const haystack = [0, 1, { foo: 'bar' }]; objectScan(['[*]', '[*].foo'])(haystack); // => [ [ 2, 'foo' ], [ 2 ], [ 1 ], [ 0 ] ] ```
#### useArraySelector Type: `boolean`
Default: `true` When set to `false`, no array selectors should be used in any needles and arrays are automatically traversed. Note that the results still include the array selectors. _Examples_:
['a', 'b.d'] (automatic array traversal) ```js const haystack = [{ a: 0 }, { b: [{ c: 1 }, { d: 2 }] }]; objectScan(['a', 'b.d'], { joined: true, useArraySelector: false })(haystack); // => [ '[1].b[1].d', '[0].a' ] ```
[''] (top level array matching) ```js const haystack = [{ a: 0 }, { b: 1 }]; objectScan([''], { joined: true, useArraySelector: false })(haystack); // => [ '[1]', '[0]' ] ```
#### strict Type: `boolean`
Default: `true` When set to `true`, errors are thrown when: - a path is identical to a previous path - a path invalidates a previous path - a path contains consecutive recursions _Examples_:
['a.b', 'a.b'] (identical) ```js const haystack = []; objectScan(['a.b', 'a.b'], { joined: true })(haystack); // => 'Error: Redundant Needle Target: "a.b" vs "a.b"' ```
['a.{b,b}'] (identical, same needle) ```js const haystack = []; objectScan(['a.{b,b}'], { joined: true })(haystack); // => 'Error: Redundant Needle Target: "a.{b,b}" vs "a.{b,b}"' ```
['a.b', 'a.**'] (invalidates previous) ```js const haystack = []; objectScan(['a.b', 'a.**'], { joined: true })(haystack); // => 'Error: Needle Target Invalidated: "a.b" by "a.**"' ```
['**.!**'] (consecutive recursion) ```js const haystack = []; objectScan(['**.!**'], { joined: true })(haystack); // => 'Error: Redundant Recursion: "**.!**"' ```
### Search Context A context can be passed into a search invocation as a second parameter. It is available in all callbacks and can be used to manage state across a search invocation without having to recompile the search. By default all matched keys are returned from a search invocation. However, when it is not _undefined_, the context is returned instead. _Examples_:
['**.{c,d,e}'] (sum values) ```js const haystack = { a: { b: { c: 2, d: 11 }, e: 7 } }; objectScan(['**.{c,d,e}'], { joined: true, filterFn: ({ value, context }) => { context.sum += value; } })(haystack, { sum: 0 }); // => { sum: 20 } ```
## Examples More extensive examples can be found in the tests.
['a.*.f'] (nested) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['a.*.f'], { joined: true })(haystack); // => [ 'a.e.f' ] ```
['*.*.*'] (multiple nested) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['*.*.*'], { joined: true })(haystack); // => [ 'a.e.f', 'a.b.c' ] ```
['a.*.{c,f}'] (or filter) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['a.*.{c,f}'], { joined: true })(haystack); // => [ 'a.e.f', 'a.b.c' ] ```
['a.*.{c,f}'] (or filter, not joined) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['a.*.{c,f}'])(haystack); // => [ [ 'a', 'e', 'f' ], [ 'a', 'b', 'c' ] ] ```
['*.*[*]'] (list filter) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['*.*[*]'], { joined: true })(haystack); // => [ 'a.h[1]', 'a.h[0]' ] ```
['*[*]'] (list filter, unmatched) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['*[*]'], { joined: true })(haystack); // => [] ```
['**'] (star recursion) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['**'], { joined: true })(haystack); // => [ 'k', 'a.h[1]', 'a.h[0]', 'a.h', 'a.e.f', 'a.e', 'a.b.c', 'a.b', 'a' ] ```
['++.++'] (plus recursion) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['++.++'], { joined: true })(haystack); // => [ 'a.h[1]', 'a.h[0]', 'a.h', 'a.e.f', 'a.e', 'a.b.c', 'a.b' ] ```
['**.f'] (star recursion ending in f) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['**.f'], { joined: true })(haystack); // => [ 'a.e.f' ] ```
['**[*]'] (star recursion ending in array) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['**[*]'], { joined: true })(haystack); // => [ 'a.h[1]', 'a.h[0]' ] ```
['a.*,!a.e'] (exclusion filter) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['a.*,!a.e'], { joined: true })(haystack); // => [ 'a.h', 'a.b' ] ```
['**.(^[bc]$)'] (regex matching) ```js const haystack = { a: { b: { c: 'd' }, e: { f: 'g' }, h: ['i', 'j'] }, k: 'l' }; objectScan(['**.(^[bc]$)'], { joined: true })(haystack); // => [ 'a.b.c', 'a.b' ] ```
## Edge Cases Top level object(s) are matched by the empty needle `''`. This is useful for matching objects nested in arrays by setting `useArraySelector` to `false`. To match the actual empty string as a key, use `(^$)`. Note that the empty string does not work to match top level objects with [_.get](https://lodash.com/docs/#get) or [_.set](https://lodash.com/docs/#set). _Examples_:
[''] (match top level objects in array) ```js const haystack = [{}, {}]; objectScan([''], { joined: true, useArraySelector: false })(haystack); // => [ '[1]', '[0]' ] ```
[''] (match top level object) ```js const haystack = {}; objectScan([''], { joined: true })(haystack); // => [ '' ] ```
['**.(^$)'] (match empty string keys) ```js const haystack = { '': 0, a: { '': 1 } }; objectScan(['**.(^$)'])(haystack); // => [ [ 'a', '' ], [ '' ] ] ```
## Internals Conceptually this package works as follows: 1. During initialization the needles are parsed and built into a search tree. Various information is pre-computed and stored for every node. Finally the search function is returned. 2. When the search function is invoked, the input is traversed simultaneously with the relevant nodes of the search tree. Processing multiple search tree branches in parallel allows for a single traversal of the input. Having a separate initialization stage allows for a performant search and significant speed ups when applying the same search to different input.