1====== Strata: Structured Data Plugin====== 2 3The strata plugin allows you to add data to your pages and to query that data from your pages. This manual is split into two parts: 4 5 * the [[#quick guide]] will get you started with a few examples, 6 * the [[#reference guide]] will show you every possible option. 7 8 9====== Quick Guide ====== 10 11The quick guide will get you up and running with some examples of how to enter and query. More advanced uses are discussed in the reference guide. 12 13A good way to get more experienced is to add some simple data to your wiki, and start querying it. Most error messages are descriptive enough to get some idea of what went wrong. 14 15 16===== Data Block ===== 17 18Data entry is done with ''<data>'' tags. The following example is a data block for Jane Doe. The block is meant to add some extra data to the page it is on (we assume it is on the page ''persons:jane_doe''). The example shows you how to add simple values, how to declare a class, and how to use types. 19 20<code> 21<data person> 22Full Name: Jane Maria Doe 23Birthday [date]: 1982-7-23 24</data> 25</code> 26 27**Simple Values**: You add simple values to the data block by adding a line like ''field: value''. The field and value are sometimes called the predicate and object respectively. 28 29**Classes**: You can add one or more classes to a data block by placing them in the opening tag. Classes are separated by spaces, so class names declared in this way can not contain spaces. (Note that declaring a class name is effectively the same as adding a ''is a: person'' field-value pair to the data block.) 30 31**Types**: You can add a [[#types|type]] to use by putting the type between ''['' and '']'' after the field name. Types determine how the data is displayed, and how it is stored. 32 33The same example, but extended with more features: 34<code> 35<data person> 36-- Simple field-value pairs 37Full Name: Jane Maria Doe 38Address: 39 40-- Types and Type Hint 41Birthday [date]: 1982-7-23 42Birthplace [page::places]: Springfield 43 44-- Multiple values 45Contact [link]*: j.doe@example.com, http://www.facebook.com/Jane-Doe 46Contact [link]: jane.doe@workmail.com 47</data> 48</code> 49 50**Empty values**: Any field that doesn't have a value is ignored. This way you can quickly write down some fields you want to use, but fill in their values later. 51 52**Type hints**: You can change how a [[#types|type]] behaves by adding a type hint. Type hints are added by appending them to the type with ''::''. For example ''[page::places]'' uses the page type, and will try to resolve values without an explicit namespace as if they were in the ''places:'' namespace. For a list of types and their hints, see [[#Types]]. 53 54**Multiple Values**: You can have multiple values with a field. Do this by either putting a ''*'' after the field (or after the type, if it has any), or by simply adding the field multiple times. 55 56**Comments** All lines that start with double dashes (i.e., ''%%--%%'') are ignored. Note that your comments can still be read by anyone viewing the source of the wiki page. 57 58 59===== Tables and Lists ===== 60 61Queries are written inside ''<table>'' or ''<list>'' tags. You query the data by describing what pattern the data should fit. A simple example that produces a table of all persons and their birthday would be described as follows: 62 63<code> 64<table ?p "Person" ?b "Birthday"> 65?p is a: person 66?p Birthday [date]: ?b 67?b < 1990-1-1 68</table> 69</code> 70 71**Patterns**: You can use variables and literals to describe what data you want to match. The patterns should be written down in lines, with each line formatted like ''subject field: value''. 72 73For example, ''?p is a: person'' will match any subject that has field ''is a'' and value ''person'' to variable ''?p''. 74 75Variables are indicated with the ''?''. You can use a variable in any spot (except types or type hints). For example ''?p ?k [date]: 1982-7-23'' to find out who has what relation to the date 1982-7-23. 76 77Literals can be written down verbatim, except for subject literals. These should be enclosed in ''%%[[%%'' and ''%%]]%%''. For example ''%%[[persons:jane_doe]] Address: ?a%%'' to get the address associated with Jane Doe. 78 79**Types**: In a query, you can use [[#types]]. You can use types for fields and values, and you can use them in the opening tag. Types are 'sticky': if you put ''?p Birthday [date]: ?b'' the date type will automatically stick to the ''?b'' variable (you could have achieved the same with ''?p Birthday: ?b [date]''). 80 81**Comparisons**: You can use normal operators (e.g, ''<'', ''>'', ''>='', ''%%<=%%'', ''='', ''!='') to compare values. A variable's type will be taken into account for the comparison. See [[#Comparison Operators]] for more information. 82 83You can only compare variables that are used in a pattern. 84 85**Captions**: You can put captions in the opening tag to change the caption of the column. Captions are not displayed by lists, but are still used to add names to the filtering and sorting interface. 86 87<code> 88<table ?p "Person" ?address@count "# of Addresses" ?address> 89?p is a: person 90 91optional { 92 ?p Address: ?address 93} 94 95minus { 96 ?p Contact: ?c 97} 98 99group { 100 ?p 101} 102sort { 103 ?address (desc) 104} 105ui { 106 Person { 107 filter: select 108 } 109 Address { 110 filter: text 111 } 112} 113</table> 114</code> 115 116**Aggregates**: Variables can have multiple values (usually through grouping). You can apply an aggregate function to the variable's values by adding it to any variable in the opening tag with ''@''. For example ''?address@count'' will apply the [[#aggregates|count]] aggregate to the values in ''?address''. 117 118For more on aggregates, see [[#Aggregates]]. 119 120**Optional matches**: Normally, all patterns must be matched for the results to be shown. You can use an ''optional'' block to indicate that some of the patterns are optional, and need not be matched for the results to be shown. All patterns in an optional block must match for the optional block to be used. If any pattern in the block doesn't match, none of the patterns in the block will be used. 121 122You can have multiple optional blocks. You can even have optional blocks inside optional blocks. 123 124**Exclusions**: With the ''minus'' block, you can declare that the data is not allowed to match certain patterns. In this case, the results are not allowed to have contact information. 125 126**Grouping**: By adding a ''group'' block zero or more variables can be grouped. This means that all results that have the same value for the grouped variable will be merged, and the ungrouped variables will contain multiple values. You can name one variable per line. If the ''group'' is empty //all// results will be merged into a single result. 127 128**Sorting**: By adding ''sort'' you can define one or more variables to sort on. You can name one variable per line, and include a direction with ''(ascending)'' or ''(descending)'' (or their short-hands ''(asc)'' and ''(desc)''). 129 130**User Interface**: By adding ''ui'' you can define how and if the user interface allows filtering (and sorting) on the client. See [[#User Interface]] for details. 131 132**Comments**: As with a [[#data block]], you can use ''%%--%%'' at the start of a line to add comments. 133 134**Caching**: By default, the results you see on the page will be cached. So if you edit other pages, you'll need to refresh the page with the list yourself, or add ''%%~~NOCACHE~~%%'' to force dokuwiki to rerender. 135 136 137====== Reference Guide ====== 138 139The reference guide is split up into four sections: 140 * [[#Data Entry]] 141 * [[#Query Language]] 142 * [[#Query Results]] 143 * [[#User Interface]] 144 * [[#Types & Aggregates]] 145 146 147===== Data Entry ===== 148 149Entering data is done with the ''<data>'' syntax. A data block defines data that is associated with the page the data block is written on. 150 151What follows is a generic pattern of the syntax of data entry 152<code> 153<data class1 class2 classN #fragment identifier> 154Field [type::hint]*: value1, value2, valueN 155</data> 156</code> 157 158== Field-Value pairs == 159 160The simplest form of data entry. Fields and values are also called predicates and objects. It is possible to leave out the value, then the field-value pair will not be stored, but you can easily fill in the missing value later. 161 162 * Field names can contain any character except the following: '':'' ''('' '')'' ''['' '']'' ''{'' ''}'' ''<'' ''>'' ''|'' ''~'' ''!'' ''@'' ''#'' ''$'' ''%'' ''^'' ''&'' ''*'' ''?'' ''='' ''%%"%%'' 163 * Values can contain any character, but values in a list of multiple values can't contain '','' 164 165You can add multiple values in a single line by adding an asterisk after the type (or after the key, if it has no type). The values should be separated by '',''. Note that you can also add multiple values just by writing multiple lines with the same key. 166 167There is a single magic value to indicate the empty value: ''%%[[]]%%'' This token can be used with any type, but is especially useful with the [[#types|ref]] and [[#types|page]] types as it will create a link to the page the data block is on. 168 169== Classes == 170 171As a convenience, you can attach one or more classes to the data by putting them in the opening: ''<data **person**>''. To add multiple classes, separate them with a space. 172 173Classes are not handled specially. This way of adding classes to the data is merely a convenience. You can achieve the same by adding values to field ''is a''. For example ''<data person>'' can be achieved by a line of ''is a: person'' in the data entry. 174 175 * Class names in the header can contain any characters except spaces, ''#'', or ''>'' 176 177== Entry Title == 178 179Normally, the ''entry title'' field is automatically generated for any data block. This field is used when displaying values with the [[#types|ref]] type. 180 181The field is generated from the fragment identifier if it is available. If the block has no fragment identifier, the page title is used as entry title. If you want to override the entry title of a data block, you can do so by adding the ''entry title'' field yourself. 182 183== Field Types == 184 185You can add a [[#types|type]] to a field by putting the type between ''['' and '']'' and placing it after the field name. This will change how the field's values are stored and displayed. Note that the declared type is only used during entry, that is, the type is not stored. You can declare a different type when [[#Querying Data]]. 186 187You can add a type hint to any type you use. You do so by adding the type hint to the type with a ''::'' marker: ''[type::hint]''. Usually, type hints change the way a type displays data. See [[#types]] for possible types and hints. 188 189== Comments == 190 191You can add comments by starting a line with ''%%--%%''. Comments are not used nor are they displayed. Note that anyone that can do 'view source' can read your comments. 192 193 194==== Data Fragments ==== 195 196Instead of associating data directly with the page, you can put it in a fragment. A fragment is a piece of data that is not directly associated with the page itself, but instead is associated with part of the page. 197 198A data fragment is not implicitly associated with the page it is defined on. If you want to add such a relation, you need to do this yourself. Note that the ''%%[[]]%%'' value can be used with the [[#types|ref]] or [[#types|page]] type to refer to the current page. 199 200== Fragment Identifiers == 201 202A data block is associated with a fragment simply by adding a fragment identifier to the block's opening tag: ''<data #fragment identifier>''. Fragment identifiers are used in much the same way as a page's sections. 203 204 * Fragment Identifiers can contain any character except ''>'' 205 206 207==== Split Data Entries ==== 208 209Sometimes, it makes sense to have all data associated with a single page, but defined in multiple data blocks throughout the page. This is possible by simply splitting the data blocks into multiple blocks. 210 211Note that the [[#classes]] do not have to be repeated. If you want to split a fragment data block, you have to add the same [[#fragment identifier]] to every data block of the fragment. 212 213 214===== Query Language ===== 215 216Querying data is done through the ''<table>'' and ''<list>'' syntax. A table or list block defines what data to query, and how to display it. This part of the reference is about querying the data. The [[#Query Results]] section discusses how to control the display of data (this includes sorting, grouping, and aggregation). 217 218The following sections contain short samples, each of these samples is situated inside a table or list block. It is possible to enclose the whole of the query (not including sorting, grouping, or other [[#Query Results]] related blocks) in a ''query'' block as a convenience. 219 220The query blocks are [[#optional]], [[#minus]], [[#union]]. 221 222 223==== Patterns ==== 224 225Patterns are the basic building block of all queries. They are constructed according to the following format: 226 227 subject predicate: object 228 229You can use variables, indicated by starting with ''?'', and literals in any position. 230 231 * Variables can contain any character except spaces or '':'' ''('' '')'' ''['' '']'' ''{'' ''}'' ''<'' ''>'' ''|'' ''~'' ''!'' ''@'' ''#'' ''$'' ''%'' ''^'' ''&'' ''*'' ''?'' ''='' ''%%"%%'' 232 * Subject literals must be enclosed in ''%%[[%%'' and ''%%]]%%'', and are interpreted as if it were a wiki link 233 * Predicate literals can contain any character except '':'' and ''['' 234 * Object literals can contain any character 235 236You can refer to 'the current page' with ''%%[[]]%%''. This can be used to write queries that get information related to the page, but defined on other pages. 237 238 239==== Typing ==== 240 241You can use types to make sure the data is interpreted in the correct way, and to create a better looking result. 242 243 subject ?predicate [type::hint]: ?object [type::hint] 244 245[[#Types]] can only be added to variables. A variable in the subject position will always be typed as [[#types|ref]]. Literals can't be typed, but will be interpreted according to the variables they are used with. 246 247Types are 'sticky'. This means that the first mentioned type for a certain variable will stick to the variable for the whole query. You can override a specific use, but without an explicit type a variable will be of the type first attached to it. 248 249Types are propagated according to the following rules: 250 * variables in the subject position are always of type [[#type|ref]] (and the ref type will stick) 251 * The first explicit mention of a type for a variable will stick that type to the variable 252 * unless the object is explicitly typed, a typed predicate will propagate its type to the object 253 254 255==== Filters ==== 256 257You can use simple filters to refine any matches from a pattern. 258 259 left > right 260 261It is possible to use both variables and literals for left and right, but there must be at least one variable present. You can only use variables that are used in a pattern in the same block or inner blocks (with the exception of [[#minus]] blocks, which don't bind any variables). 262 263=== Comparison Operators === 264 265The following filters apply to all types of data: 266^ Generic ^^ 267^ Filter ^ Name ^ 268| ''!='' | Not equal | 269| ''='' | Equal | 270 271These filters only make sense on numerical values: 272^ Numerical ^^ 273^ Filter ^ Name ^ 274| ''>'' | Greater than | 275| ''>='' | Greater than or equal | 276| ''<'' | Less than | 277| ''%%<=%%'' | Less than or equal | 278 279These filters (usually) only make sense on textual values: 280^ Textual ^^ 281^ Filter ^ Name ^ 282| ''~'' | Contains | 283| ''!~'' | Does not contain | 284| ''%%^~%%'' | Starts with | 285| ''%%!^~%%'' | Does not start with | 286| ''$~'' | Ends with | 287| ''!$~'' | Does not end with | 288| ''~>'' | In wiki namespace | 289| ''!~>'' | Not in wiki namespace | 290 291The ''~>'' and ''!~>'' operators are special cases of ''^~'' and ''!^~'' that use the [[#types|text]] type when comparing a left-sided variable to a right-sided literal, regardless of the variable's type. The use of these two operators makes most sense when comparing variables with the [[#types|ref]] or [[#types|page]] types. 292 293 294==== Optional ==== 295 296Optional blocks can be used to optionally match extra patterns. 297 298 optional { 299 ... 300 } 301 302An optional block must contain at least a [[#patterns|pattern]]. Filters and other query blocks are also possible. An optional block is only used if all patterns in the block match. 303 304 305==== Union ==== 306 307You can tell the wiki to combine two patterns. 308 309 union { 310 { 311 ... 312 } 313 { 314 ... 315 } 316 } 317 318An union block can contain more than two options, but must have at least two. All options must contain at least a pattern, but can contain filters and query blocks as well. 319 320 321==== Minus ==== 322 323A minus block is used to exclude all results for which the patterns in the minus block match. 324 325 minus { 326 ... 327 } 328 329A minus block must contain at least a pattern, but can contain filters and other query blocks. 330 331 332===== Query Results ===== 333 334This section describes the options you have to control the output of the query. 335 336The query result blocks are: [[#sorting|sort]], [[#grouping|group]], [[#variable projection|fields]] and [[#Considering Fields|consider]]. 337 338 339==== Sorting ==== 340 341You can sort on one or more variables. 342 343 sort { 344 ?variable (direction) 345 } 346 347The sort block takes a single variable per line, with an optional direction between parenthesis. Both full (ascending and descending) and abbreviated (asc and desc) are usable. 348 349 350==== Grouping ==== 351 352Grouping on one or more variables allows you to create overviews. 353 354 group { 355 ?variable 356 } 357 358Grouping allows you to collapse multiple results into a single result. All results that have the same value for all variables mentioned in the group block will be merged into a single result. Any variable in the merged result that is not mentioned in the group block will contain multiple values. 359 360 361==== Variable Projection ==== 362 363To define the variables to display, you can use the shorthand or the long syntax: 364 365 <table ?variable@aggregate(hint) [type::hint] "Caption"> 366 367 fields { 368 ?variable@aggregate(hint) [type::hint]: Caption 369 } 370 371All elements except the variable itself are optional. If left out, a reasonable guess or default is used. 372 373 * The default aggregate is to use no aggregation 374 * The default type is the type associated with the variable in the query 375 * The default caption is the variable name with a capital first letter 376 377Any variables not mentioned in the projection are left out of consideration for determining what the results are. This might create a problem where simple results from a complex query seem incomplete, in that case try [[#Considering fields]]. 378 379 380==== Aggregation Functions ==== 381 382Aggregation functions are used to process a variables captured values before display. These functions can be used for things like counting, summing up or reducing the values to only the unique values. 383 384See [[#Aggregates]] for all possible aggregates and their hints. 385 386 387==== Considering fields ==== 388 389If a variable is not mentioned as one of the displayed fields, it will be ignored. You can hint that some field needs to be considered, but not displayed. 390 391 consider { 392 ?variable 393 } 394 395All variables mentioned will be considered to be relevant, even if they are not displayed. Since the queries use so called 'set semantics', results that are equivalent will be reduced to a single result. This can be prevented by declaring additional variables to be considered; results are only equivalent if all displayed and all considered fields are equal. 396 397===== User Interface ===== 398The ''%%ui%%'' block controls the interactive user interface of tables and lists. It can be used to make columns sortable and filterable. A ''%%ui%%'' block looks as follows: 399 400<code> 401ui { 402 ui: generic 403 sort: default 404 filter: text 405} 406</code> 407 408The properties that can be specified, are explained in the following subsections. 409 410Note: Users that have JavaScript disabled will not benefit from the user interface settings. 411==== UI ==== 412The property ''%%ui%%'' specifies which user interface is shown and can have the values: ''%%none%%'', ''%%generic%%'', and ''%%table%%'' (only for tables). When the ''%%ui%%'' block is present, this property defaults to ''%%table%%'' for tables and ''%%generic%%'' otherwise. However, when no ''%%ui%%'' block is present, tables do have a ''%%table%%'' ui by default, since it is very subtle, but lists have no interactive ui. 413==== Sorting ==== 414Using ''%%sort%%'' you can specify whether all columns are sortable (''%%default%%'') or not (''%%none%%''). Instead of the default sorting order, you can also choose ''%%left to right%%'' or ''%%right to left%%'' sorting. 415==== Filtering ==== 416Using ''%%filter%%'' you can specify whether and how columns are filterable: 417 * ''text'': a text input field. It filters rows that have a value that contains the filter string (substring matching). E.g.: filtering for //David// will match //10 David Street//. 418 * ''%%select%%'': a drop down select box filter, which contains all possible values. It filters rows that have exactly the value that is selected (exact matching). 419 * ''%%prefix select%%'': a drop down select box filter, which contains all possible values. It filters rows that start with the value that is selected (prefix matching). E.g.: if the values //EN//, //%%EN-US%%//, and //%%EN-GB%%// are present, filtering for //EN// will match all three values. 420 * ''%%suffix select%%'': a drop down select box filter, which contains all possible values. It filters rows that end with the value that is selected (suffix matching). E.g.: if the values //10 David Street// and //David Street// are present, filtering for //David Street// will match both values. 421 * ''%%none%%'': no filtering. 422By default, columns are not filterable. 423 424==== Per column configuration ==== 425The properties ''%%sort%%'' and ''%%filter%%'' are applied on each column. It is also possible to set these properties differently for some columns, as follows: 426<code> 427<table> 428fields { 429 Person 1: ?p1 430 Relation: ?r 431 Person 2: ?p2 432} 433ui { 434 filter: text 435 Relation { 436 filter: select 437 sort: none 438 } 439} 440?p1 is a: person 441?p2 is a: person 442?p1 ?r: ?p2 443</table> 444</code> 445 446Here, we use a block ''%%Relation%%'' to set different properties for the column named ''%%Relation%%'' (this name is specified in the ''%%fields%%'' group). Thus, all columns are sortable and have text-based filters, except the column ''%%Relation%%'', which is not sortable and has a select-based filter. 447 448Besides identifying columns by name, they can also be identified by number. For example, column ''%%Relation%%'' is also column ''%%#2%%''. 449 450When multiple columns have the same name, settings are applied to all columns that have the given name. To identify specific columns, index them by number. If you use both a block with a name (e.g. ''%%Relation%%'') and one with a number (e.g. ''%%#2%%''), then all settings specified in the numbered block will override the ones specified in the named block. (If you name a column ''%%#2%%'', then ''%%#2%%'' is considered a name, not a number.) 451 452=== Alternative column configuration === 453Instead of using a block for each column, it is also possible to set all ''%%sort%%'' or ''%%filter%%'' values at once, but this is---in general---less readable than specifying a block for each column. Using this style, the previous example will become as follows: 454<code> 455<table> 456fields { 457 Person 1: ?p1 458 Relation: ?r 459 Person 2: ?p2 460} 461ui { 462 filter*: text, select, text 463 sort*: no, yes, no 464} 465?p1 is a: person 466?p2 is a: person 467?p1 ?r: ?p2 468</table> 469</code> 470By adding a ''%%*%%'' to the property name, we indicate that we are setting multiple columns at once. We must set all columns (e.g.: here, precisely three values must be given). However, we can leave some value the default by giving them the empty string: 471<code> 472<table> 473fields { 474 Person 1: ?p1 475 Relation: ?r 476 Person 2: ?p2 477} 478ui { 479 -- We specify the second column as block, so don't care about the values now 480 filter*: text, , text 481 sort*: no,,no 482 Relation { 483 filter: select 484 sort: no 485 } 486} 487?p1 is a: person 488?p2 is a: person 489?p1 ?r: ?p2 490</table> 491</code> 492 493==== UI with Aggregates ==== 494 495In general, the UI is quite intuitive, but combining it with aggregates might give unexpected results (unless you use a table, in which case no special handling is needed). The example below shows the column ''%%address%%'' twice: once the actual values and once the number of values. Because the UI creates filters per column, only one filter will be created for the ''%%address%%'' column. This filter filters both on addresses and number of addresses per person. 496 497<code> 498<list ?p "Person" ?address@count "# of Addresses" ?address> 499?p is a: person 500 501optional { 502 ?p Address: ?address 503} 504 505group { 506 ?p 507} 508sort { 509 ?address (desc) 510} 511ui { 512 filter: text 513} 514</list> 515</code> 516 517As shown below, the above query can easily be converted to one which lists each column only once and, therefore, does have separate filters for the address count and the addresses. 518 519<code> 520<list ?p "Person" ?address-to-count@count "# of Addresses" ?address> 521?p is a: person 522 523optional { 524 ?p Address: ?address 525 ?p Address: ?address-to-count 526 ?address = ?address-to-count 527} 528 529group { 530 ?p 531} 532sort { 533 ?address (desc) 534} 535ui { 536 filter: text 537} 538</list> 539</code> 540===== Types & Aggregates ===== 541 542Types and aggregates are used to control how data is stored and displayed. 543 544Types are used with data entry to store data in the correct format. Types with queries are used for handling comparisons, and to determine how the data should be displayed. 545 546Aggregates are used to process values after a query, but before they are displayed. 547 548 549==== Types ==== 550 551Types are normally indicated by putting them between ''['' and '']''. Type hints are noted after the type itself in the following format: ''[type::hint]'' 552 553~~INFO:stratatypes~~ 554 555 556==== Aggregates ==== 557 558Aggregates are used on displays of variables. They are attached to the variable with ''@'', for example: ''?x@unique'' applies the unique aggregate to the values of variables ''?x''. Aggregates can be passed a hint by adding the hint between parenthesis: ''?variable@aggregate(hint)''. 559 560~~INFO:strataaggregates~~ 561 562 563===== Caching ===== 564 565By default, strata does not disable caching. This can result in pages with queries not updating after you edit data somewhere else. 566 567If you edit other pages, you'll need to refresh the page with the list yourself, or add ''%%~~NOCACHE~~%%'' to the page with the query to force dokuwiki to refresh the page every time. 568