1====== Strata: Structured Data Plugin======
2
3The strata plugin allows you to add data to your pages and to query that data from your pages. This manual is split into two parts:
4
5  * the [[#quick guide]] will get you started with a few examples,
6  * the [[#reference guide]] will show you every possible option.
7
8
9====== Quick Guide ======
10
11The quick guide will get you up and running with some examples of how to enter and query. More advanced uses are discussed in the reference guide.
12
13A good way to get more experienced is to add some simple data to your wiki, and start querying it. Most error messages are descriptive enough to get some idea of what went wrong.
14
15
16===== Data Block =====
17
18Data entry is done with ''<data>'' tags. The following example is a data block for Jane Doe. The block is meant to add some extra data to the page it is on (we assume it is on the page ''persons:jane_doe''). The example shows you how to add simple values, how to declare a class, and how to use types.
19
20<code>
21<data person>
22Full Name: Jane Maria Doe
23Birthday [date]: 1982-7-23
24</data>
25</code>
26
27**Simple Values**: You add simple values to the data block by adding a line like ''field: value''. The field and value are sometimes called the predicate and object respectively.
28
29**Classes**: You can add one or more classes to a data block by placing them in the opening tag. Classes are separated by spaces, so class names declared in this way can not contain spaces. (Note that declaring a class name is effectively the same as adding a ''is a: person'' field-value pair to the data block.)
30
31**Types**: You can add a [[#types|type]] to use by putting the type between ''['' and '']'' after the field name. Types determine how the data is displayed, and how it is stored.
32
33The same example, but extended with more features:
34<code>
35<data person>
36-- Simple field-value pairs
37Full Name: Jane Maria Doe
38Address:
39
40-- Types and Type Hint
41Birthday [date]: 1982-7-23
42Birthplace [page::places]: Springfield
43
44-- Multiple values
45Contact [link]*: j.doe@example.com, http://www.facebook.com/Jane-Doe
46Contact [link]: jane.doe@workmail.com
47</data>
48</code>
49
50**Empty values**: Any field that doesn't have a value is ignored. This way you can quickly write down some fields you want to use, but fill in their values later.
51
52**Type hints**: You can change how a [[#types|type]] behaves by adding a type hint. Type hints are added by appending them to the type with ''::''. For example ''[page::places]'' uses the page type, and will try to resolve values without an explicit namespace as if they were in the ''places:'' namespace. For a list of types and their hints, see [[#Types]].
53
54**Multiple Values**: You can have multiple values with a field. Do this by either putting a ''*'' after the field (or after the type, if it has any), or by simply adding the field multiple times.
55
56**Comments** All lines that start with double dashes (i.e., ''%%--%%'') are ignored. Note that your comments can still be read by anyone viewing the source of the wiki page.
57
58
59===== Tables and Lists =====
60
61Queries are written inside ''<table>'' or ''<list>'' tags. You query the data by describing what pattern the data should fit. A simple example that produces a table of all persons and their birthday would be described as follows:
62
63<code>
64<table ?p "Person" ?b "Birthday">
65?p is a: person
66?p Birthday [date]: ?b
67?b < 1990-1-1
68</table>
69</code>
70
71**Patterns**: You can use variables and literals to describe what data you want to match. The patterns should be written down in lines, with each line formatted like ''subject field: value''.
72
73For example, ''?p is a: person'' will match any subject that has field ''is a'' and value ''person'' to variable ''?p''.
74
75Variables are indicated with the ''?''. You can use a variable in any spot (except types or type hints). For example ''?p ?k [date]: 1982-7-23'' to find out who has what relation to the date 1982-7-23.
76
77Literals can be written down verbatim, except for subject literals. These should be enclosed in ''%%[[%%'' and ''%%]]%%''. For example ''%%[[persons:jane_doe]] Address: ?a%%'' to get the address associated with Jane Doe.
78
79**Types**: In a query, you can use [[#types]]. You can use types for fields and values, and you can use them in the opening tag. Types are 'sticky': if you put ''?p Birthday [date]: ?b'' the date type will automatically stick to the ''?b'' variable (you could have achieved the same with ''?p Birthday: ?b [date]'').
80
81**Comparisons**: You can use normal operators (e.g, ''<'', ''>'', ''>='', ''%%<=%%'', ''='', ''!='') to compare values. A variable's type will be taken into account for the comparison. See [[#Comparison Operators]] for more information.
82
83You can only compare variables that are used in a pattern.
84
85**Captions**: You can put captions in the opening tag to change the caption of the column. Captions are not displayed by lists, but are still used to add names to the filtering and sorting interface.
86
87<code>
88<table ?p "Person" ?address@count "# of Addresses" ?address>
89?p is a: person
90
91optional {
92  ?p Address: ?address
93}
94
95minus {
96  ?p Contact: ?c
97}
98
99group {
100  ?p
101}
102sort {
103  ?address (desc)
104}
105ui {
106  Person {
107    filter: select
108  }
109  Address {
110    filter: text
111  }
112}
113</table>
114</code>
115
116**Aggregates**: Variables can have multiple values (usually through grouping). You can apply an aggregate function to the variable's values by adding it to any variable in the opening tag with ''@''. For example ''?address@count'' will apply the [[#aggregates|count]] aggregate to the values in ''?address''.
117
118For more on aggregates, see [[#Aggregates]].
119
120**Optional matches**: Normally, all patterns must be matched for the results to be shown. You can use an ''optional'' block to indicate that some of the patterns are optional, and need not be matched for the results to be shown. All patterns in an optional block must match for the optional block to be used. If any pattern in the block doesn't match, none of the patterns in the block will be used.
121
122You can have multiple optional blocks. You can even have optional blocks inside optional blocks.
123
124**Exclusions**: With the ''minus'' block, you can declare that the data is not allowed to match certain patterns. In this case, the results are not allowed to have contact information.
125
126**Grouping**: By adding a ''group'' block zero or more variables can be grouped. This means that all results that have the same value for the grouped variable will be merged, and the ungrouped variables will contain multiple values. You can name one variable per line. If the ''group'' is empty //all// results will be merged into a single result.
127
128**Sorting**: By adding ''sort'' you can define one or more variables to sort on. You can name one variable per line, and include a direction with ''(ascending)'' or ''(descending)'' (or their short-hands ''(asc)'' and ''(desc)'').
129
130**User Interface**: By adding ''ui'' you can define how and if the user interface allows filtering (and sorting) on the client. See [[#User Interface]] for details.
131
132**Comments**: As with a [[#data block]], you can use ''%%--%%'' at the start of a line to add comments.
133
134**Caching**: By default, the results you see on the page will be cached. So if you edit other pages, you'll need to refresh the page with the list yourself, or add ''%%~~NOCACHE~~%%'' to force dokuwiki to rerender.
135
136
137====== Reference Guide ======
138
139The reference guide is split up into four sections:
140  * [[#Data Entry]]
141  * [[#Query Language]]
142  * [[#Query Results]]
143  * [[#User Interface]]
144  * [[#Types & Aggregates]]
145
146
147===== Data Entry =====
148
149Entering data is done with the ''<data>'' syntax. A data block defines data that is associated with the page the data block is written on.
150
151What follows is a generic pattern of the syntax of data entry
152<code>
153<data class1 class2 classN #fragment identifier>
154Field [type::hint]*: value1, value2, valueN
155</data>
156</code>
157
158== Field-Value pairs ==
159
160The simplest form of data entry. Fields and values are also called predicates and objects. It is possible to leave out the value, then the field-value pair will not be stored, but you can easily fill in the missing value later.
161
162  * Field names can contain any character except the following: '':'' ''('' '')'' ''['' '']'' ''{'' ''}'' ''<'' ''>'' ''|'' ''~'' ''!'' ''@'' ''#'' ''$'' ''%'' ''^'' ''&'' ''*'' ''?'' ''='' ''%%"%%''
163  * Values can contain any character, but values in a list of multiple values can't contain '',''
164
165You can add multiple values in a single line by adding an asterisk after the type (or after the key, if it has no type). The values should be separated by '',''. Note that you can also add multiple values just by writing multiple lines with the same key.
166
167There is a single magic value to indicate the empty value: ''%%[[]]%%'' This token can be used with any type, but is especially useful with the [[#types|ref]] and [[#types|page]] types as it will create a link to the page the data block is on.
168
169== Classes ==
170
171As a convenience, you can attach one or more classes to the data by putting them in the opening: ''<data **person**>''. To add multiple classes, separate them with a space.
172
173Classes are not handled specially. This way of adding classes to the data is merely a convenience. You can achieve the same by adding values to field ''is a''. For example ''<data person>'' can be achieved by a line of ''is a: person'' in the data entry.
174
175  * Class names in the header can contain any characters except spaces, ''#'', or ''>''
176
177== Entry Title ==
178
179Normally, the ''entry title'' field is automatically generated for any data block. This field is used when displaying values with the [[#types|ref]] type.
180
181The field is generated from the fragment identifier if it is available. If the block has no fragment identifier, the page title is used as entry title. If you want to override the entry title of a data block, you can do so by adding the ''entry title'' field yourself.
182
183== Field Types ==
184
185You can add a [[#types|type]] to a field by putting the type between ''['' and '']'' and placing it after the field name. This will change how the field's values are stored and displayed. Note that the declared type is only used during entry, that is, the type is not stored. You can declare a different type when [[#Querying Data]].
186
187You can add a type hint to any type you use. You do so by adding the type hint to the type with a ''::'' marker: ''[type::hint]''. Usually, type hints change the way a type displays data. See [[#types]] for possible types and hints.
188
189== Comments ==
190
191You can add comments by starting a line with ''%%--%%''. Comments are not used nor are they displayed. Note that anyone that can do 'view source' can read your comments.
192
193
194==== Data Fragments ====
195
196Instead of associating data directly with the page, you can put it in a fragment. A fragment is a piece of data that is not directly associated with the page itself, but instead is associated with part of the page.
197
198A data fragment is not implicitly associated with the page it is defined on. If you want to add such a relation, you need to do this yourself. Note that the ''%%[[]]%%'' value can be used with the [[#types|ref]] or [[#types|page]] type to refer to the current page.
199
200== Fragment Identifiers ==
201
202A data block is associated with a fragment simply by adding a fragment identifier to the block's opening tag: ''<data #fragment identifier>''. Fragment identifiers are used in much the same way as a page's sections.
203
204  * Fragment Identifiers can contain any character except ''>''
205
206
207==== Split Data Entries ====
208
209Sometimes, it makes sense to have all data associated with a single page, but defined in multiple data blocks throughout the page. This is possible by simply splitting the data blocks into multiple blocks.
210
211Note that the [[#classes]] do not have to be repeated. If you want to split a fragment data block, you have to add the same [[#fragment identifier]] to every data block of the fragment.
212
213
214===== Query Language =====
215
216Querying data is done through the ''<table>'' and ''<list>'' syntax. A table or list block defines what data to query, and how to display it. This part of the reference is about querying the data. The [[#Query Results]] section discusses how to control the display of data (this includes sorting, grouping, and aggregation).
217
218The following sections contain short samples, each of these samples is situated inside a table or list block. It is possible to enclose the whole of the query (not including sorting, grouping, or other [[#Query Results]] related blocks) in a ''query'' block as a convenience.
219
220The query blocks are [[#optional]], [[#minus]], [[#union]].
221
222
223==== Patterns ====
224
225Patterns are the basic building block of all queries. They are constructed according to the following format:
226
227  subject predicate: object
228
229You can use variables, indicated by starting with ''?'', and literals in any position.
230
231  * Variables can contain any character except spaces or '':'' ''('' '')'' ''['' '']'' ''{'' ''}'' ''<'' ''>'' ''|'' ''~'' ''!'' ''@'' ''#'' ''$'' ''%'' ''^'' ''&'' ''*'' ''?'' ''='' ''%%"%%''
232  * Subject literals must be enclosed in ''%%[[%%'' and ''%%]]%%'', and are interpreted as if it were a wiki link
233  * Predicate literals can contain any character except '':'' and ''[''
234  * Object literals can contain any character
235
236You can refer to 'the current page' with ''%%[[]]%%''. This can be used to write queries that get information related to the page, but defined on other pages.
237
238
239==== Typing ====
240
241You can use types to make sure the data is interpreted in the correct way, and to create a better looking result.
242
243  subject ?predicate [type::hint]: ?object [type::hint]
244
245[[#Types]] can only be added to variables. A variable in the subject position will always be typed as [[#types|ref]]. Literals can't be typed, but will be interpreted according to the variables they are used with.
246
247Types are 'sticky'. This means that the first mentioned type for a certain variable will stick to the variable for the whole query. You can override a specific use, but without an explicit type a variable will be of the type first attached to it.
248
249Types are propagated according to the following rules:
250  * variables in the subject position are always of type [[#type|ref]] (and the ref type will stick)
251  * The first explicit mention of a type for a variable will stick that type to the variable
252  * unless the object is explicitly typed, a typed predicate will propagate its type to the object
253
254
255==== Filters ====
256
257You can use simple filters to refine any matches from a pattern.
258
259  left > right
260
261It is possible to use both variables and literals for left and right, but there must be at least one variable present. You can only use variables that are used in a pattern in the same block or inner blocks (with the exception of [[#minus]] blocks, which don't bind any variables).
262
263=== Comparison Operators ===
264
265The following filters apply to all types of data:
266^ Generic ^^
267^ Filter ^ Name ^
268|  ''!=''  | Not equal |
269|  ''=''  | Equal |
270
271These filters only make sense on numerical values:
272^ Numerical ^^
273^ Filter ^ Name ^
274|  ''>''  | Greater than |
275|  ''>=''  | Greater than or equal |
276|  ''<''  | Less than |
277|  ''%%<=%%''  | Less than or equal |
278
279These filters (usually) only make sense on textual values:
280^ Textual ^^
281^ Filter ^ Name ^
282|  ''~''  | Contains |
283|  ''!~''  | Does not contain |
284|  ''%%^~%%''  | Starts with |
285|  ''%%!^~%%''  | Does not start with |
286|  ''$~''  | Ends with |
287|  ''!$~''  | Does not end with |
288|  ''~>''  | In wiki namespace |
289|  ''!~>''  | Not in wiki namespace |
290
291The ''~>'' and ''!~>'' operators are special cases of ''^~'' and ''!^~'' that use the [[#types|text]] type when comparing a left-sided variable to a right-sided literal, regardless of the variable's type. The use of these two operators makes most sense when comparing variables with the [[#types|ref]] or [[#types|page]] types.
292
293
294==== Optional ====
295
296Optional blocks can be used to optionally match extra patterns.
297
298  optional {
299    ...
300  }
301
302An optional block must contain at least a [[#patterns|pattern]]. Filters and other query blocks are also possible. An optional block is only used if all patterns in the block match.
303
304
305==== Union ====
306
307You can tell the wiki to combine two patterns.
308
309  union {
310    {
311      ...
312    }
313    {
314      ...
315    }
316  }
317
318An union block can contain more than two options, but must have at least two. All options must contain at least a pattern, but can contain filters and query blocks as well.
319
320
321==== Minus ====
322
323A minus block is used to exclude all results for which the patterns in the minus block match.
324
325  minus {
326    ...
327  }
328
329A minus block must contain at least a pattern, but can contain filters and other query blocks.
330
331
332===== Query Results =====
333
334This section describes the options you have to control the output of the query.
335
336The query result blocks are: [[#sorting|sort]], [[#grouping|group]], [[#variable projection|fields]] and [[#Considering Fields|consider]].
337
338
339==== Sorting ====
340
341You can sort on one or more variables.
342
343  sort {
344    ?variable (direction)
345  }
346
347The sort block takes a single variable per line, with an optional direction between parenthesis. Both full (ascending and descending) and abbreviated (asc and desc) are usable.
348
349
350==== Grouping ====
351
352Grouping on one or more variables allows you to create overviews.
353
354  group {
355    ?variable
356  }
357
358Grouping allows you to collapse multiple results into a single result. All results that have the same value for all variables mentioned in the group block will be merged into a single result. Any variable in the merged result that is not mentioned in the group block will contain multiple values.
359
360
361==== Variable Projection ====
362
363To define the variables to display, you can use the shorthand or the long syntax:
364
365  <table ?variable@aggregate(hint) [type::hint] "Caption">
366
367  fields {
368    ?variable@aggregate(hint) [type::hint]: Caption
369  }
370
371All elements except the variable itself are optional. If left out, a reasonable guess or default is used.
372
373  * The default aggregate is to use no aggregation
374  * The default type is the type associated with the variable in the query
375  * The default caption is the variable name with a capital first letter
376
377Any variables not mentioned in the projection are left out of consideration for determining what the results are. This might create a problem where simple results from a complex query seem incomplete, in that case try [[#Considering fields]].
378
379
380==== Aggregation Functions ====
381
382Aggregation functions are used to process a variables captured values before display. These functions can be used for things like counting, summing up or reducing the values to only the unique values.
383
384See [[#Aggregates]] for all possible aggregates and their hints.
385
386
387==== Considering fields ====
388
389If a variable is not mentioned as one of the displayed fields, it will be ignored. You can hint that some field needs to be considered, but not displayed.
390
391  consider {
392    ?variable
393  }
394
395All variables mentioned will be considered to be relevant, even if they are not displayed. Since the queries use so called 'set semantics', results that are equivalent will be reduced to a single result. This can be prevented by declaring additional variables to be considered; results are only equivalent if all displayed and all considered fields are equal.
396
397===== User Interface =====
398The ''%%ui%%'' block controls the interactive user interface of tables and lists. It can be used to make columns sortable and filterable. A ''%%ui%%'' block looks as follows:
399
400<code>
401ui {
402  ui: generic
403  sort: default
404  filter: text
405}
406</code>
407
408The properties that can be specified, are explained in the following subsections.
409
410Note: Users that have JavaScript disabled will not benefit from the user interface settings.
411==== UI ====
412The property ''%%ui%%'' specifies which user interface is shown and can have the values: ''%%none%%'', ''%%generic%%'', and ''%%table%%'' (only for tables). When the ''%%ui%%'' block is present, this property defaults to ''%%table%%'' for tables and ''%%generic%%'' otherwise. However, when no ''%%ui%%'' block is present, tables do have a ''%%table%%'' ui by default, since it is very subtle, but lists have no interactive ui.
413==== Sorting ====
414Using ''%%sort%%'' you can specify whether all columns are sortable (''%%default%%'') or not (''%%none%%''). Instead of the default sorting order, you can also choose ''%%left to right%%'' or ''%%right to left%%'' sorting.
415==== Filtering ====
416Using ''%%filter%%'' you can specify whether and how columns are filterable:
417  * ''text'': a text input field. It filters rows that have a value that contains the filter string (substring matching). E.g.: filtering for //David// will match //10 David Street//.
418  * ''%%select%%'': a drop down select box filter, which contains all possible values. It filters rows that have exactly the value that is selected (exact matching).
419  * ''%%prefix select%%'': a drop down select box filter, which contains all possible values. It filters rows that start with the value that is selected (prefix matching). E.g.: if the values //EN//, //%%EN-US%%//, and //%%EN-GB%%// are present, filtering for //EN// will match all three values.
420  * ''%%suffix select%%'': a drop down select box filter, which contains all possible values. It filters rows that end with the value that is selected (suffix matching). E.g.: if the values //10 David Street// and //David Street// are present, filtering for //David Street// will match both values.
421  * ''%%none%%'': no filtering.
422By default, columns are not filterable.
423
424==== Per column configuration ====
425The properties ''%%sort%%'' and ''%%filter%%'' are applied on each column. It is also possible to set these properties differently for some columns, as follows:
426<code>
427<table>
428fields {
429  Person 1: ?p1
430  Relation: ?r
431  Person 2: ?p2
432}
433ui {
434  filter: text
435  Relation {
436    filter: select
437    sort: none
438  }
439}
440?p1 is a: person
441?p2 is a: person
442?p1 ?r: ?p2
443</table>
444</code>
445
446Here, we use a block ''%%Relation%%'' to set different properties for the column named ''%%Relation%%'' (this name is specified in the ''%%fields%%'' group). Thus, all columns are sortable and have text-based filters, except the column ''%%Relation%%'', which is not sortable and has a select-based filter.
447
448Besides identifying columns by name, they can also be identified by number. For example, column ''%%Relation%%'' is also column ''%%#2%%''.
449
450When multiple columns have the same name, settings are applied to all columns that have the given name. To identify specific columns, index them by number. If you use both a block with a name (e.g. ''%%Relation%%'') and one with a number (e.g. ''%%#2%%''), then all settings specified in the numbered block will override the ones specified in the named block. (If you name a column ''%%#2%%'', then ''%%#2%%'' is considered a name, not a number.)
451
452=== Alternative column configuration ===
453Instead of using a block for each column, it is also possible to set all ''%%sort%%'' or ''%%filter%%'' values at once, but this is---in general---less readable than specifying a block for each column. Using this style, the previous example will become as follows:
454<code>
455<table>
456fields {
457  Person 1: ?p1
458  Relation: ?r
459  Person 2: ?p2
460}
461ui {
462  filter*: text, select, text
463  sort*: no, yes, no
464}
465?p1 is a: person
466?p2 is a: person
467?p1 ?r: ?p2
468</table>
469</code>
470By adding a ''%%*%%'' to the property name, we indicate that we are setting multiple columns at once. We must set all columns (e.g.: here, precisely three values must be given). However, we can leave some value the default by giving them the empty string:
471<code>
472<table>
473fields {
474  Person 1: ?p1
475  Relation: ?r
476  Person 2: ?p2
477}
478ui {
479  -- We specify the second column as block, so don't care about the values now
480  filter*: text, , text
481  sort*: no,,no
482  Relation {
483    filter: select
484    sort: no
485  }
486}
487?p1 is a: person
488?p2 is a: person
489?p1 ?r: ?p2
490</table>
491</code>
492
493==== UI with Aggregates ====
494
495In general, the UI is quite intuitive, but combining it with aggregates might give unexpected results (unless you use a table, in which case no special handling is needed). The example below shows the column ''%%address%%'' twice: once the actual values and once the number of values. Because the UI creates filters per column, only one filter will be created for the ''%%address%%'' column. This filter filters both on addresses and number of addresses per person.
496
497<code>
498<list ?p "Person" ?address@count "# of Addresses" ?address>
499?p is a: person
500
501optional {
502  ?p Address: ?address
503}
504
505group {
506  ?p
507}
508sort {
509  ?address (desc)
510}
511ui {
512  filter: text
513}
514</list>
515</code>
516
517As shown below, the above query can easily be converted to one which lists each column only once and, therefore, does have separate filters for the address count and the addresses.
518
519<code>
520<list ?p "Person" ?address-to-count@count "# of Addresses" ?address>
521?p is a: person
522
523optional {
524  ?p Address: ?address
525  ?p Address: ?address-to-count
526  ?address = ?address-to-count
527}
528
529group {
530  ?p
531}
532sort {
533  ?address (desc)
534}
535ui {
536  filter: text
537}
538</list>
539</code>
540===== Types & Aggregates =====
541
542Types and aggregates are used to control how data is stored and displayed.
543
544Types are used with data entry to store data in the correct format. Types with queries are used for handling comparisons, and to determine how the data should be displayed.
545
546Aggregates are used to process values after a query, but before they are displayed.
547
548
549==== Types ====
550
551Types are normally indicated by putting them between ''['' and '']''. Type hints are noted after the type itself in the following format: ''[type::hint]''
552
553~~INFO:stratatypes~~
554
555
556==== Aggregates ====
557
558Aggregates are used on displays of variables. They are attached to the variable with ''@'', for example: ''?x@unique'' applies the unique aggregate to the values of variables ''?x''. Aggregates can be passed a hint by adding the hint between parenthesis: ''?variable@aggregate(hint)''.
559
560~~INFO:strataaggregates~~
561
562
563===== Caching =====
564
565By default, strata does not disable caching. This can result in pages with queries not updating after you edit data somewhere else.
566
567If you edit other pages, you'll need to refresh the page with the list yourself, or add ''%%~~NOCACHE~~%%'' to the page with the query to force dokuwiki to refresh the page every time.
568