Skip to content

Commit bd9b31b

Browse files
josdejongjosdejong
committed
Merge branch 'main' into develop
# Conflicts: # reference/functions.md
2 parents 09b7cb6 + 8f7f675 commit bd9b31b

File tree

2 files changed

+6
-1526
lines changed

2 files changed

+6
-1526
lines changed

‎README.md

Lines changed: 5 additions & 276 deletions
Original file line numberDiff line numberDiff line change
@@ -22,19 +22,16 @@ On this page:
2222

2323
- [Installation](#installation)
2424
- [Usage](#usage)
25-
- [Syntax](#syntax)
26-
- [JSON Format](#json-format)
2725
- [JavaScript API](#javascript-api)
2826
- [Command line interface (CLI)](#command-line-interface-cli)
29-
- [Gotchas](#gotchas)
3027
- [Development](#development)
31-
- [Motivation](#motivation)
3228
- [License](#license)
3329

3430
External pages:
3531

36-
- [Function reference](reference/functions.md)
37-
- [Test Suite](test-suite/README.md)
32+
- [JSON Query Documentation](https://jsonquerylang.org/docs/)
33+
- [JSON Query Function reference](https://jsonquerylang.org/reference/)
34+
- [JSON Query Test Suite](test-suite/README.md)
3835

3936
## Installation
4037

@@ -105,235 +102,7 @@ const result = jsonquery(data, 'times(3)', options)
105102
// [3, 6, 9]
106103
```
107104

108-
## Syntax
109-
110-
The `jsonquery` language looks quite similar to JavaScript and other JSON query languages. This makes it easy to learn. When writing a query, you compose a ["pipe"](https://medium.com/@efeminella/the-pipe-operator-a-glimpse-into-the-future-of-functional-javascript-7ebb578887a4) or a ["chain"](https://en.wikipedia.org/wiki/Method_chaining) of operations to be applied to the data. It resembles chaining like in [Lodash](https://lodash.com/docs/4.17.15#chain) or just [in JavaScript](https://medium.com/backticks-tildes/understanding-method-chaining-in-javascript-647a9004bd4f) itself using methods like `map` and `filter`.
111-
112-
Queries are written in a plain text format which is compact and easy to read for humans. The text format is parsed into an intermediate JSON format which is easy to operate on programmatically. This JSON format is executed by the query engine.
113-
114-
The text format has functions, operators, property getters, pipes to execute multiple queries in series, and objects to execute multiple queries in parallel or transform the input. For example:
115-
116-
```text
117-
filter(.age >= 18) | sort(.age)
118-
```
119-
120-
The text format can be converted (back and forth) into a JSON format consisting purely of composed function calls. A function call is described by an array containing the function name followed by its arguments, like `[name, arg1, arg2, ...]`. Here is the JSON equivalent of the previous example:
121-
122-
```json
123-
[
124-
"pipe",
125-
["filter", ["gte", ["get", "age"], 18]],
126-
["sort", ["get", "age"]]
127-
]
128-
```
129-
130-
The JSON format is mostly used under the hood. It allows for easy integrations like a GUI or executing the query in a different environment or language without having to implement a parser for the text format. Read more in the [JSON Format](#json-format) section.
131-
132-
### Syntax overview
133-
134-
The following table gives an overview of the JSON query text format:
135-
136-
| Type | Syntax | Example |
137-
|-------------------------|------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
138-
| [Function](#functions) | `name(argument1, argument2, ...)` | `sort(.age, "asc")` |
139-
| [Operator](#operators) | `(left operator right)` | `filter(.age >= 18)` |
140-
| [Pipe](#pipes) | <code>query1 &#124; query2 &#124; ...</code> | <code>sort(.age) &#124; pick(.name, .age)</code> |
141-
| [Object](#objects) | `{ prop1: query1, prop2: query2, ... }` | `{ names: map(.name), total: sum() }` |
142-
| [Array](#arrays) | `[ item1, item2, ... ]` | `[ "New York", "Atlanta" ]` |
143-
| [Property](#properties) | `.prop1`<br/>`.prop1.prop2`<br/>`."prop1"`<br/>`get("prop1", "prop2")`<br/>`get()` | `.age`<br/>`.address.city`<br/>`."first name"`<br/>`get("address", "city")`<br/>`get()` |
144-
| [String](#values) | `"string"` | `"Hello world"` |
145-
| [Number](#values) | A floating point number | `2.4` |
146-
| [Boolean](#values) | `true` or `false` | `true` |
147-
| [null](#values) | `null` | `null` |
148-
149-
The syntax is explained in detail in the following sections. The examples are based on querying the following data:
150-
151-
```json
152-
[
153-
{ "name": "Chris", "age": 23, "address": { "city": "New York" } },
154-
{ "name": "Emily", "age": 19, "address": { "city": "Atlanta" } },
155-
{ "name": "Joe", "age": 32, "address": { "city": "New York" } },
156-
{ "name": "Kevin", "age": 19, "address": { "city": "Atlanta" } },
157-
{ "name": "Michelle", "age": 27, "address": { "city": "Los Angeles" } },
158-
{ "name": "Robert", "age": 45, "address": { "city": "Manhattan" } },
159-
{ "name": "Sarah", "age": 31, "address": { "city": "New York" } }
160-
]
161-
```
162-
163-
### Functions
164-
165-
Function calls have the same syntax as in most programming languages:
166-
167-
```text
168-
name(argument1, argument2, ...)
169-
```
170-
171-
The following example will `sort` the data in ascending order, sorted by the property `age`.
172-
173-
```text
174-
sort(.age, "asc")
175-
```
176-
177-
Important to understand is that the functions are executed as a method in a chain: the sorting is applied to the data input, and forwarded to the next method in the chain (if any). The following example first filters the data, and next sorts it:
178-
179-
```text
180-
filter(.age >= 21) | sort(.age, "asc")
181-
```
182-
183-
See section [Function reference](reference/functions.md) for a detailed overview of all available functions and operators.
184-
185-
### Operators
186-
187-
JSON Query supports all basic operators. Operators must be wrapped in parentheses `(...)`, must have both a left and right hand side, and do not have precedence since parentheses are required. The syntax is:
188-
189-
```text
190-
(left operator right)
191-
```
192-
193-
The following example tests whether a property `age` is greater than or equal to `18`:
194-
195-
```text
196-
(.age >= 18)
197-
```
198-
199-
Operators are for example used to specify filter conditions:
200-
201-
```text
202-
filter(.age >= 18)
203-
```
204-
205-
When composing multiple operators, it is necessary to use parentheses:
206-
207-
```text
208-
filter((.age >= 18) and (.age <= 65))
209-
```
210-
211-
See section [Function reference](reference/functions.md) for a detailed overview of all available functions and operators.
212-
213-
### Pipes
214-
215-
A _pipe_ is a series of multiple query operations separated by a pipe character `|`. The syntax is:
216-
217-
```text
218-
query1 | query2 | ...
219-
```
220-
221-
The entries in the pipeline are executed one by one, and the output of the first is the input for the next. The following example will first filter the items of an array that have a nested property `city` in the object `address` with the value `"New York"`, and next, sort the filtered items by the property `age`:
222-
223-
```text
224-
filter(.address.city == "New York") | sort(.age)
225-
```
226-
227-
### Objects
228-
229-
An _object_ is defined as a regular JSON object with a property name as key, and query as value. Objects can be used to transform data or to execute multiple queries in parallel.
230-
231-
```text
232-
{ prop1: query1, prop2: query2, ... }
233-
```
234-
235-
The following example will transform the data by mapping over the items of the array and creating a new object with properties `firstName` and `city` for every item:
236-
237-
```text
238-
map({
239-
firstName: .name,
240-
city: .address.city
241-
})
242-
```
243-
244-
The following example runs multiple queries in parallel. It outputs an object with properties `names`, `count`, and `averageAge` containing the results of their query: a list with names, the total number of array items, and the average value of the properties `age` in all items:
245-
246-
```text
247-
{
248-
names: map(.name),
249-
count: size(),
250-
averageAge: map(.age) | average()
251-
}
252-
```
253-
254-
A property can be unquoted when it only contains characters `a-z`, `A-Z`, `_` and `$`, and all but the first character can be a number `0-9`. When the property contains other characters, like spaces, it needs to be enclosed in double quotes and escaped like JSON keys:
255-
256-
```text
257-
{
258-
"first name": map(.name)
259-
}
260-
```
261-
262-
### Arrays
263-
264-
Arrays are defined like JSON arrays: enclosed in square brackets, with items separated by a comma:
265-
266-
```text
267-
[query1, query2, ...]
268-
```
269-
270-
Arrays can for example be used for the operators `in` and `not in`:
271-
272-
```text
273-
filter(.city in ["New York", "Atlanta"])
274-
```
275-
276-
### Properties
277-
278-
An important feature is the property getter. It allows to get a property from an object:
279-
280-
```text
281-
.age
282-
```
283-
284-
A nested property can be retrieved by specifying multiple properties. The following path for example describes the value of a nested property `city` inside an object `address`:
285-
286-
```text
287-
.address.city
288-
```
289-
290-
A property can be unquoted when it only contains characters `a-z`, `A-Z`, `_` and `$`, and all but the first character can be a number `0-9`. When the property contains other characters, like spaces, it needs to be enclosed in double quotes and escaped like JSON keys:
291-
292-
```text
293-
."first name"
294-
```
295-
296-
To get the current value itself, use the function `get` without arguments:
297-
298-
```text
299-
get()
300-
```
301-
302-
And, alternatively to the dot notation, the function `get` can be used for properties and nested properties too:
303-
304-
```text
305-
get("age")
306-
get("address", "city")
307-
```
308-
309-
310-
### Values
311-
312-
JSON Query supports the following primitive values, the same as in [JSON](https://www.json.org): `string`, `number`, `boolean`, `null`.
313-
314-
| Type | Example |
315-
|---------|-------------------------------------------------------------------|
316-
| string | `"Hello world"`<br/>`"Multi line text\nwith \"quoted\" contents"` |
317-
| number | `42`<br/>`2.74`<br/>`-1.2e3`<br/> |
318-
| boolean | `true`<br/>`false` |
319-
| null | `null` |
320-
321-
## JSON format
322-
323-
The text format describe above can be converted into an intermediate JSON format consisting purely of composed function calls and vice versa. A function call is described by an array containing the function name followed by its arguments, like `[name, arg1, arg2, ...]`. The following table gives an overview of the text format and the equivalent JSON format.
324-
325-
| Type | Text format | JSON format |
326-
|----------|----------------------------------------------|---------------------------------------------------------------------------|
327-
| Function | `name(argument1, argument2, ...)` | `["name", argument1, argument2, ...]` |
328-
| Operator | `(left operator right)` | `["operator", left, right]` |
329-
| Pipe | <code>query1 &#124; query2 &#124; ...</code> | `["pipe", query1, query2, ...]` |
330-
| Object | `{ prop1: query1, prop2: query2, ... }` | `["object", { "prop1": query1, "prop2": query2, ... }]` |
331-
| Array | `[ item1, item2, ... ]` | `["array", item1, item2, ... ]` |
332-
| Property | `.prop1`<br/>`.prop1.prop2`<br/>`."prop1"` | `["get", "prop1"]`<br/>`["get", "prop1", "prop2"]`<br/>`["get", "prop1"]` |
333-
| String | `"string"` | `"string"` |
334-
| Number | A floating point number | A floating point number |
335-
| Boolean | `true` or `false` | `true` or `false` |
336-
| null | `null` | `null` |
105+
Documentation on the syntax of JSON Query and all supported functions can be found on the website: https://jsonquerylang.org/docs/.
337106

338107
## JavaScript API
339108

@@ -356,7 +125,7 @@ jsonquery(data: JSON, query: string | JSONQuery, options: JSONQueryOptions) : JS
356125
Here:
357126

358127
- `data` is the JSON document that will be queried, often an array with objects.
359-
- `query` is a JSON document containing a JSON query, either the text format or the parsed JSON format.
128+
- `query` is a JSON document containing a [JSON query](https://jsonquerylang.org/docs/), either the text format or the parsed JSON format.
360129
- `options` is an optional object that can contain the following properties:
361130
- `functions` is an optional map with custom function creators. A function creator has optional arguments as input and must return a function that can be used to process the query data. For example:
362131

@@ -626,18 +395,6 @@ $ cat users.json | jsonquery 'sort(.age)'
626395
$ cat users.json | jsonquery 'sort(.age)' > output.json
627396
```
628397

629-
## Gotchas
630-
631-
The JSON Query language has some gotchas. What can be confusing at first is to understand how data is piped through the query. A traditional function call is for example `max(myValues)`, so you may expect to have to write this in JSON Query like `["max", "myValues"]`. However, JSON Query has a functional approach where we create a pipeline like: `data -> max -> result`. So, you will have to write a pipe which first gets this property and next calls the function max: `.myValues | max()`.
632-
633-
Another gotcha is that unlike some other query languages, you need to use the `map` function to pick some properties out of an array _for every item_ in the array. When you're just picking a few fields without renaming them, you can use the function `pick` too, which is more concise.
634-
635-
```
636-
.friends | { firstName: .name, age: .age } WRONG
637-
.friends | map({ firstName: .name, age: .age }) RIGHT
638-
.friends | pick(.name, .age) RIGHT
639-
```
640-
641398
## Development
642399

643400
### JavaScript
@@ -656,34 +413,6 @@ npm run build-and-test
656413

657414
Note that a new package is published on [npm](https://www.npmjs.com/package/@jsonquerylang/jsonquery) and [GitHub](https://github.com/jsonquerylang/jsonquery/releases) on changes pushed to the `main` branch. This is done using [`semantic-release`](https://github.com/semantic-release/semantic-release), and we do not use the `version` number in the `package.json` file. A changelog can be found by looking at the [releases on GitHub](https://github.com/jsonquerylang/jsonquery/releases).
658415

659-
### Implement in a new language
660-
661-
Support for JSON Query language can be implemented in new programming languages. Implementing the query engine is most straight forward: this boils down to implementing each of the functions (`sort`, `filter`, `groupBy`, etc.), and creating a compiler which can go through a JSON Query like `["sort", ["get", "name"], "desc"]` look up the function `sort`, and pass the arguments to it. Implementing a parser and stringifier is a bit more work, but the parser and stringifier of the JavaScript implementation can be used as a reference.
662-
663-
There is a JSON based Test Suite available that can be used to ensure that your implementation matches the behavior of the reference implementation, see: [Test Suite](test-suite/README.md).
664-
665-
## Motivation
666-
667-
There are many powerful query languages out there, so why the need to develop `jsonquery`? There are a couple of reasons for this.
668-
669-
1. **Syntax**
670-
671-
Most JSON query languages have a syntax that is simple for very basic queries, but complex for more advanced queries. Their syntax is typically very compact but includes special characters and notations (like `@`, `$`, `|`, `?`, `:`, `*`, `&`), almost feeling like Regex which is infamously hard to read. The syntax is hard to remember unless you use the query language on a daily basis.
672-
673-
2. **Size**
674-
675-
Most of the JSON query languages are quite big when looking at the bundle size. This can make them unsuitable for use in a web application where every kilobyte counts.
676-
677-
3. **Expressiveness**
678-
679-
The expressiveness of most query languages is limited. Since a long time, my favorite JSON query language is JavaScript+Lodash because it is so flexible. The downside however is that it is not safe to store or share queries written in JavaScript: executing arbitrary JavaScript can be a security risk.
680-
681-
4. **Parsable**
682-
683-
When a query language is simple to parse, it is easy to write integrations and adapters for it. For example, it is possible to write a visual user interface to write queries, and the query language can be implemented in various environments (frontend, backend).
684-
685-
The `jsonquery` language is inspired by [JavaScript+Lodash](https://jsoneditoronline.org/indepth/query/10-best-json-query-languages/#javascript), [JSON Patch](https://jsonpatch.com/), and [MongoDB aggregates](https://www.mongodb.com/docs/manual/aggregation/). It is basically a JSON notation to describe making a series of function calls. It has no magic syntax except for the need to be familiar with JSON, making it flexible and easy to understand. The library is extremely small thanks to smartly utilizing built-in JavaScript functions and the built-in JSON parser, requiring very little code to make the query language work.
686-
687416
## License
688417

689418
Released under the [ISC license](LICENSE.md).

0 commit comments

Comments
 (0)