1====== JSON Data Plugin ====== 2 3---- plugin ---- 4description: Build JSON database inside DokuWiki page and use the data in the page 5author : Janez Paternoster 6email : janez.paternoster@siol.net 7type : syntax, action, helper, remote 8lastupdate : 2024-03-11 9compatible : Hogfather, Igor, Jack Jackrum, Kaos 10depends : 11conflicts : 12similar : jsoneditor, jsontable, jsongendoc, struct, data, strata 13tags : data, json, database, template, xmlrpc, listing, tables 14 15downloadurl: https://gitlab.com/dokuwiki-json/json/-/archive/master/json-master.zip 16bugtracker : https://gitlab.com/dokuwiki-json/json/-/issues 17sourcerepo : https://gitlab.com/dokuwiki-json/json 18donationurl: https://paypal.me/jnz022 19 20screenshot_img: https://gitlab.com/dokuwiki-json/json/-/raw/master/demo/screenshot.png 21---- 22 23===== Installation ===== 24 25Search and install the plugin using the [[plugin:extension|Extension Manager]]. Refer to [[:Plugins]] on how to install plugins manually. 26 27You may want to install other related plugins: 28 * [[plugin:jsoneditor|JSON editor plugin]] 29 * [[plugin:jsontable|JSON table plugin]] 30 * [[plugin:jsongendoc|JSON generate document plugin]] 31 32 33===== Description ===== 34 35With JSON Data Plugin you can build [[https://www.json.org/|JSON]] database inside DokuWiki page. JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is a [[wp>Document-oriented database]], so no SQL is used. 36 37To add JSON database into the wiki page write: 38<code> 39<json path=person>{ 40 "firstName": "James", 41 "lastName": "Smith" 42}</json> 43</code> 44 45With multiple ''<json>'' elements inside the page database is generated internally. There may also be external data sources. 46 47Data from internal database can be used on the page, for example: 48<code> 49Person described here is %$person.firstName% %$person.lastName%. 50</code> 51 52will produce: "Person described here is James Smith." 53 54JSON data can be used in the following ways: 55 * Extract specific JSON **basic** element. 56 * Extract specific JSON object element as a **list**. 57 * Extract specific JSON array element as a **table**. 58 * Print specific part of data as JSON code. 59 * Show or hide sections of DokuWiki page, based on value of specific JSON element. 60 * As a datasource for a javascript widget. 61 * JSON data can also be edited and safely stored back into the dokuwiki page via ajax call. 62 63 64===== Demo ===== 65 66There is a [[https://dokuwiki-json-demo.1001beauty.si/|DokuWiki JSON Demo Server]] with JSON database integrated into DokuWiki. Also source code of this plugin contains demo for JSON Data Definition and Usage. You can copy the contents of the demo files into your DokuWiki and experiment with them. 67 68 69===== Support ===== 70 71For issues or just questions use [[https://gitlab.com/dokuwiki-json/json/-/issues|Issues]] on Gitlab. Please don't email directly. 72 73 74===== JSON Data Definition Syntax ===== 75 76Each page must define it's JSON data. Data are inline or are loaded from external (text) files on each page refresh. Only those data are loaded, which are defined inside a page. Data are defined with one or more '%%<json>%%' elements: 77 78''<json ''//attributes//''>''//inline_json//''</json>'' 79 80When the DokuWiki page is served, there is internally a variable, which is an empty object on the beginning. For example ''%%json_database = {}%%''. After the above first ''<json>'' element is rendered, variable is something like this: ''%%json_database = {"person": {"firstname": "James", "lastName": "Smith"}}%%''. After each subsequent ''%%<json>%%'' element data is added to that database. ''path'' attribute specifies, where in the ''json_database'' data are appended or replaced. 81 82 83==== Inline JSON ==== 84 85Valid [[https://www.json.org/|JSON]] data can be located between %%<json ...>%% and %%</json>%% tags. If both, inline data and data referenced by 'src' attribute are specified, then inline data have precedence. Inline data will replace overlapped data referenced by 'src' attribute. 86 87 88==== Attribute 'id' ==== 89 90''id'' attribute must be specified, when we want to access this json element from somewhere else or if we want to write inline JSON data via ajax call. It must be unique on one page. 91 92 93==== Attribute 'path' ==== 94 95''path'' attribute specifies, where in the 'json_database' data will be added. For example ''path=person1.address'' will write data into ''json_database.person1.address''. ''.'' (dot) is used as a delimiter between tokens. If tokens contains spaces, then they must be inside quotes. 96 97If token is number, then it points to n-th array member. ''0'' points to first array member and so on. Special token ''_FIRST_'' or ''_LAST_'' points to the first or to the last member. 98 99If there are already data on the specified path, then new data will recursively replace overlapped original data. (For detailed rules see php function [[phpfn>array_replace_recursive]].) This is true, if the following two modifiers are not used. 100 101Modifier **-**: if ''-'' (minus) is set in the beginning of the path attribute, then original data on the path will be erased before new data will be written. For example ''path=-person1.address''. 102 103Modifier **[]**: open square bracket immediately followed by closing square bracket at the end of the path attribute means, that data will be appended (pushed) to the specified path on the 'json_database'. For example ''path=persons[]'' will put the data in the ''json_database.persons'' array in the next free index. 104 105If path attribute is not specified, data is combined with the 'json_database' directly. 106 107 108==== Attribute 'src' ==== 109 110''src'' attribute contains a [[wiki:syntax#links|DokuWiki link]] to the external data. External data is a text file. It may be document from the same dokuwiki or it can be located anywhere on the net. 111<code> 112src=path:to:remote_document 113src=https://www.example.com/file.json 114src=path:to:remote_document#specific_element 115src=persons:person* 116src='{"JSON": "data", "can clone": %$pre-defined.data%}' 117</code> 118 119External data may be a pure JSON file, it begins with ''['' or ''{''. 120 121External data may be a text file with %%<json>%% elements inside - remote DokuWiki page. Remote DokuWiki page then first loads data according to the rules inside its %%<json>%% elements, builds its own database and then passes the generated database into our document. Please note, that %%<json>%% elements inside remote dokuwiki page may also contain 'src' attribute. In this case data from that 'src' are also loaded. Data are loaded recursively. How deep recursion goes is controlled in configuration setting 'src_recursive'. 122 123External data may be a specific %%<json>%% element from remote dokuwiki page. In that case %%<json>%% element must have an 'id' attribute. It is referenced as in third example above. If target %%<json>%% element have a 'src' attribute, then data is loaded recursively. 124 125Wildcards may be may be used in 'src' attribute. See fourth example above. In that case multiple files are read and integrated into our database. See [[phpfn>glob()|glob() function]]. 126 127Fifth example above shows, that 'src' attribute can not only be used for the file path, it can also contain JSON data. This way some already defined data from page can be "cloned" and reused. Similar as in [[#data_snippets_inside_inline_json|Data snippets inside inline JSON]]. JSON data are automatically recognized, if 'src' attribute contains: ''['', ''{'' or ''%'' on the beginning and '']'', '']'' or ''%'' on the end. 128 129 130==== Attribute 'src_ext' ==== 131 132This attribute is similar to 'src'. But the link to remote data is defined externally. Value of 'src_ext' contains a name, which must begin with ''json_''. For example, 'our_document' contains: 133<code> 134<json src_ext=json_my_data></json> 135</code> 136 137In some other page we have a link to 'our_document' with one (or more) extra argument, like this: 138<code> 139[[somewhere:our_document?json_my_data=path:to:remote_document]] 140</code> 141 142'our_document' will be loaded with data as specified in [[wp>Query string]]. Rules for the file link from Query string are the same as for the 'src' attribute. We can put multiple arguments separated by ''&'' in Query string. If we want to specify specific %%<json>%% element, then we can't use ''#'', because it is reserved. We can use ''%23'' instead. 143 144If besides 'src_ext' also 'src' attribute is specified, then 'src_ext' has precedence. If link for 'src_ext' is not defined in query string, then link from 'src' is used. 145 146 147==== Attribute 'src_path' ==== 148 149If attribute ''src_path'' is specified, then only part of database referenced by 'src' attribute is used. Path on 'src' specified by 'src_path' must exist. Rules for 'src_path' are the same as for 'path', except modifiers are not used. 150 151 152==== Attribute 'src_path_ext' ==== 153 154This attribute is similar to 'src_path', but the value is defined externally in query string. It works the same way as attribute 'src_ext'. 155 156 157==== Attribute 'display' ==== 158 159This attribute controls, which part of %%<json>%% element is rendered on html page. Multiple data can be displayed with jQuery UI Tabs widget. Attribute 'display' is a comma separated list of tokens. Default value for this attribute is specified in configuration setting 'json_display'. Tokens: 160 * 'original' - display data already in database combined with data from 'src' attribute. But not yet combined with 'inline' data. 161 * 'inline' - display inline_data from %%<json>inline_data</json>%% element in textarea. 162 * 'combined' - display original data combined with 'inline' data. 163 * 'log' - display log of data sources. 164 * 'error' - display log of data sources only, of there are errors. 165 166There are also tokens 'orig-hidden', 'inl-hidden' and 'comb-hidden', which are similar to above three tokens. Each renders a hidden html 'div' element, which can be used for passing data to javascript widgets. 167 168By default, if no 'display' attribute, only tabs menu will be shown with two tabs: 'inline' and 'log'. Both tabs will be collapsed. This can be changed in configuration settings. 169 170If we want to add some tokens to default 'display' options, we can use comma in front of our token(s) in 'display' attribute. 171 172If we want only to embed data into document and hide the element, we may set display='error' to show element only if there are errors or set display=%%''%% to unconditionally hide the element. 173 174If we want to show contents of specific tab, then asterisk (*) should be added after the corresponding token. If 'display' contains only one token, followed by asterisk (*), then tabs menu will be hidden. For example 'display=combined*' will only show JSON data without the tabs menu. 175 176If 'inline' token is specified, then inline data are shown in textarea. That data can be edited and saved to the document. When data is first changed, then 'Save' button is shown in tab area. There are some rules, which may prevent the saving. First, %%<json>%% element must have unique 'id' specified. Second, if someone else changed the data since last page reload, then data can not be written. It is necessary to reload the page first (in another browser tab?) and re-enter our data. If data is successfully saved, then 'Save' button disappears from tab area. 177 178 179==== Attribute 'archive' ==== 180 181This attribute enables us to make archive of the JSON database loaded by 'src' or 'src_ext' attribute. Archive is stored into DokuWiki page itself as JSON string. This action is triggered, when user presses a button. 182 183We can specify 'archive' attribute for each %%<jsonxxx>%% element on the page. We can specify 'archive=make' or 'archive=disable'. If at least one %%<jsonxxx>%% element has 'archive' attribute equal to 'make' or 'disable', then button 'Archive JSON database' appears on the top of DokuWiki page. When user presses that button, following procedure is triggered: 184 - Verify for errors: user permission, file unmodified, lock, etc. 185 - Search DokuWiki page and find all ''%%<jsonxxx archive=make ...>%%'' elements. Replace them with ''%%<jsonxxx archive='%%/JSON_encoded_string/%%' ...>%%''. 186 - Find all ''%%<jsonxxx archive=disable src=... src_ext=...>%%'' elements and replace them with ''%%<jsonxxx archive_disabled=disable src_disabled=... src_ext_disabled=...>%%''. This disables 'src' and 'src_ext' attributes. 187 - Save the DokuWiki page. 188 189When DokuWiki page is archived, then 'archive' attribute of some or all %%<jsonxxx>%% elements contains JSON database. Data is then read from 'archive' attribute. 'src' and 'scr_ext' attributes are then ignored. Inline JSON data remain unchanged. 190 191 192===== JSON Data Usage Syntax ===== 193 194Basic pattern for render JSON data in dokuwiki is: ''%%%$ ... %%%''. JSON data can be displayed in multiple different ways, for example: simple string, as link, list of properties, table, json code, etc. Detailed description of pattern: 195<code> 196%$path [(table_row_filter)] {header} #format# (filter)% 197</code> 198Each of: ''[[#path]]'', ''[[#table_row_filter]]'', ''[[#header]]'', ''[[#format]]'', ''[[#filter]]'' is optional. Order of the brackets is important. If square brackets are used, then table will be rendered. Else if curly brackets are used, then list will be rendered. Otherwise single variable will be rendered. If special character must be used inside the pattern, then use [[https://ascii.cl/htmlcodes.htm|HTML code]]. Use ''%'' instead of ''%'', for example. 199 200 201==== path ==== 202 203'path' specifies part of the 'json_database', which will be rendered. ''.'' (dot) is used as a delimiter between tokens. Path may contain spaces, no quotes should be used. Following characters are not allowed inside path: '[]{}#()'. There are two special tokens, which may be used as part of the path: ''__FIRST__'' selects the first element inside the array and ''__LAST__'' selects the last element inside he array. 204 205 206==== table_row_filter ==== 207 208Square brackets will render table. //table_row_filter// inside brackets is optional. It will display each row of the table, if filter is matched. Rules are the same as for filter below. 209 210 211==== header ==== 212 213//header// is a comma separated list of ''key:value'' pairs, where ''key'' is a header description and ''value'' is a path to variable. It is used to render a header in a table or in a list of properties. 214 215In //header// it is also possible to define tooltips, which can be displayed on mouse hover on specific table row or table cell. Tooltip is defined, if ''key'' is prepended with special string ''_tooltip_''. 216 217Example header for two column table and with tooltip on second column: ''%%"Header 1":data.1, "Header 2":data.2, "_tooltip_Header 2":data.2.tooltip%%''. This example can also be used for a list of two properties with tooltip on second property. 218 219If we want to use one tooltip for whole table row, then we use just ''_tooltip_'' string for the ''key''. 220 221 222==== format ==== 223 224Format may be applied on any variable, list item or table column. It can render value of the variable to format other than text. Supported formats are: 225 * ''code'' - render variable as json code. 226 * ''headern'' - render header, where n is the number from 1-5 for header level. 227 * ''link'' - render as internal or external or windows share link. If variable is external link, it must have protocol specified, for example '%%http://...%%'. Variable may also have title specified, for example 'some:link|Some Title'. 228 * ''email'' - render as email address. Variable may have title specified. 229 * ''media?L200x300'' - render as internal or external media file. ''?'' and parameters are optional. First letter for parameter must be 'l' for left, 'c' for center or 'r' for right position of the media file. Other part of parameter is width x height. Parameter may also be just 'linkonly'. Variable may have title specified. 230 * ''rss n nosort reverse author date details'' - Render as rss. Rules are the same as for [[wiki:syntax#rss_atom_feed_aggregation|DokuWiki syntax]]. 231 * ''ejs?''//''template''// - Use [[https://ejs.co/|Embedded JavaScript templating]]. EJS will render data according to the //''template''// with usage of the powerful javaScript language. This option must be enabled in configuration settings, it is disabled by default. //''template''// is a string designed according to the rules for ejs, for example ''<$=d.toUpperCase()$>''. Variable (data) is passed to javaScript as ''d''. Because of string limitations not all characters may be passed to the template directly: characters ''%'', ''#'', '':'', '','', ''<%='' and ''%>'' must be written as ''%'', ''#'', '':'', '','', ''<$='' and ''$>''. 232 233Format may be used to render single variable or specific member of list or specific table column. For list or table, format must be a comma separated list of key:value pairs, where key is header description (same as in //header//) and value is format. 234 235 236==== filter ==== 237 238Filter may be applied inside brackets on any variable, list or table. If it evaluates to true, contents of the variable will be shown, otherwise not. Filter consists of one or multiple conditions separated by keywords ''or'' or ''and''. Condition is a path compared to some value(string, numeric, boolean or null). Comparison operators are: ''=='', ''!='', ''<'', ''>'', ''%%<=%%'' or ''>=''. Comparison operator and value may also be omitted. There is no precedence or brackets possible. Quotes should not be used. 239 240 241==== Examples ==== 242 243^ Type ^ Example ^ Description ^ 244| **Basic variable** | ''%%%$path.to.var%%%'' | If type of variable is string or number or boolean, it just prints it out. If it is null, it prints '(null)' by default. This can be changed in Configuration Settings. If type of the variable is array or object, this syntax prints printable members of the array or object. If there is nothing printable it prints '(array)'. | 245| **Use filter** | ''%%%$path.to.var (path.to.cond1 >= a and path.to.cond1 <b)%%%'' | Print variable only, if filter evaluates to true. | 246| **Print JSON data** | ''%%%$path.to.var #code#%%%'' | Print JSON raw data. | 247| **Variable is a link** | ''%%%$path.to.var #link#%%%'' | Variable will render as link with optional title. | 248| **List all properties** | ''%%%$path.to.var {}%%%'' | All properties of a variable will be displayed in two column table. | 249| **List specific properties** | ''%%%$path.to.var {"Property 1": subvar.path, "Property 2": subvar2.path2}%%%'' | Similar as above, but only specific properties with custom name will be printed | 250| **List specific properties with format** | ''%%%$path.to.var { ... } #"Property 1": email#%%%'' | Similar as above, but "Property 1" will be rendered as email. | 251| **Simple table** | ''%%%$path.to.var []%%%'' | If path.to.var is well formed double array, then table without header will be rendered. If path.to.var is more complex, then result may be mixed. | 252| **Table with header** | ''%%%$path.to.var [] {"Column 1": subvar.path, "Column 2": subvar2.path2}%%%'' | path.to.var should be array with similar members. //header// describes header names and members to be rendered in the table. If only blank ''{}'' is used, then header will be generated automatically. | 253| **Table with format** | ''%%%$path.to.var [] { ... } #"Column 2": link}#%%%'' | Same as above, but "Column 2" will be rendered as a link. | 254| **Table with filter** | ''%%%$path.to.var [(subvar.in.row == something)] { ... }%%%'' | Same as above, but only those rows will be rendered, which match the filter. | 255| **Conditionally hide section** | ''%%%$-start (path.to.cond1 == true)%%%''//''dokuwiki-contents''//''%%%$end%%%'' | Evaluate //filter// inside brackets. If true, then any //dokuwiki-contents// will be normally rendered. If false, then //dokuwiki-contents// will be hidden and if //dokuwiki-contents// contains title, it won't be shown in TOC. | 256| **Conditionally hide inline text** | ''%%%$-starti ( ... )% ... %$end%%%'' | Same as above, but for inline DokuWiki contents. Mind extra 'i' in 'starti'. | 257 258 259===== Data snippets inside inline JSON ===== 260 261<code> 262<json path=renault>{ 263 "cars": %$cars[(row_filter)](filter)% 264}</json> 265</code> 266Inside inline json we can use ''%%%$ path [( row_filter )]( filter )%%%'' syntax, which will be replaced by data from existing JSON database. ''[( row_filter )]'' and ''( filter )'' are optional and can be set according to the same rules, as described above. 267 268If data is not defined, then ''null'' will be used. 269 270If data is string, then double quotes must be used. Strings can also be concatenated. For example, ''%%"%$path.name%, %$path.age% years"%%'' can be used for json string concatenated from two predefined strings. 271 272 273==== Insert referenced data into array elements ==== 274 275This is advanced feature. Let's say, we have two data definitions: one for items database and one for item purchase history. For example: 276<code> 277<json path=items_db>{ 278 "apple": { "Description": "Apple", "type": "fruit" }, 279 "salad": { "Description": "Salad", "type": "vegetable" } 280}</json> 281</code> 282<code> 283<json path=items_purchased>[ 284 { "item": "apple", "quantity": 5, "date":"2019-06-22" }, 285 { "item": "salad", "quantity": 1, "date":"2019-06-28" }, 286 { "item": "apple", "quantity": 3, "date":"2019-07-10" } 287]</json> 288</code> 289 290We want to extend each item in //items_purchased// data with item type, which is available in //items_db// data definition. We can make this: 291<code> 292<json path=items_purchased_extended> 293 %$items_purchased[{item_type: items_db.{item}.type}]% 294</json> 295</code> 296And get this: 297<code> 298[{ "item": "apple", "quantity": 5, "date":"2019-06-22", "item_type": "fruit" }, 299 { "item": "salad", "quantity": 1, "date":"2019-06-28", "item_type": "vegetable" }, 300 { "item": "apple", "quantity": 3, "date":"2019-07-10", "item_type": "fruit" }] 301</code> 302 303Filters can also be used. Full definition for data snippet is: 304<code> 305%$path[(row_filter){property.path: data.path.1.{reference.path}.data.path.2, ...}](filter)% 306</code> 307 308 309===== Text macros ===== 310 311JSON define ''%%<json ...> ... </json>%%'' or extract ''%%%$ ... %%%'' patterns can be quite verbose and some lengthy attributes may repeat across multiple pages. 312 313It is possible to use macros defined by [[plugin:textinsert|textinsert Plugin]]. First install the plugin, then define some macros in it. Use ''%%#@macro_name@#%%'' patterns inside json define or extract patterns. JSON define or extract patterns are preprocessed for simple replacement of macros defined globally by textinsert Plugin. 314 315For example, if macro ''table_header'' is defined as ''%%{"Part description": part, "Quantity of parts": quantity}%%'', then you can use ''%%%$data [] #@table_header@# %%%''. 316 317 318===== Use JavaScript widgets with JSON data ===== 319 320JSON plugin has interface for other sub-plugins, which can use JSON data inside JavaScript widgets for example. It is relative simple to write such a plugin. For example [[plugin:jsoneditor|JSONEditor]] plugin uses nice [[https://json-editor.github.io/json-editor/|JSON Schema based Editor]] JavaScript library, which generates forms based on JSON Schema. It uses ''<jsoneditor>'' element, which is first rendered by JSON plugin. JSONeditor plugin has only helper.php script, which renders some additional html code into page. Data from the JSON database are already available for the widget as well as data saving mechanism. 321 322 323===== Remote access to JSON data ===== 324 325DokuWiki has a [[devel:xmlrpc|XML-RPC API]] which can be used to access/interact with your wiki from other applications. 326 327 328==== Available Functions ==== 329 330=== plugin.json.get === 331 332^ Name | **plugin.json.get** | 333^ Description | Generate JSON database on page and return data from the JSON_path. | 334^ Parameter (string) pagename | DokuWiki [[:pagename|page name]]. | 335^ Parameter (string) JSON_path | Path on the JSON database, see [[#attribute_path]]. Optional parameter, empty by default. | 336^ Parameter (boolean) addLog | If true, additional information about JSON generation will be returned. Optional parameter, false by default. | 337^ Return (object) | {'status': 'OK_or_error_description', 'data': 'JSON_data', 'log': 'JSON_loading_log' } | 338 339 340=== plugin.json.set === 341 342^ Name | **plugin.json.set** | 343^ Description | Find ''<json id=...'' element inside page and set its inline data. | 344^ Parameter (string) pagename | DokuWiki [[:pagename|page name]]. | 345^ Parameter (string) json_id | Id of the json element, see [[#attribute_id]]. | 346^ Parameter (array%%|%%string) data | Data to be put inside json element. Must be an array, empty string or valid JSON string. | 347^ Parameter (boolean) overwrite | If true, existing data will be overwritten. Optional parameter, false by default. | 348^ Return (string) | 'OK' on success or error description. | 349 350 351=== plugin.json.append === 352 353^ Name | **plugin.json.append** | 354^ Description | Find ''<json id=...'' element inside page and append data to its inline database. Inline JSON data must be an array or empty. If empty, new array will be initialized. | 355^ Parameter (string) pagename | DokuWiki [[:pagename|page name]]. | 356^ Parameter (string) json_id | Id of the json element, see [[#attribute_id]]. If empty, complete page will be treated as JSON database (page must be a JSON array, empty or non-existent). | 357^ Parameter (array%%|%%string) data | JSON Data to be appended inside json element. Must be an array or valid JSON string. | 358^ Return (string) | 'OK' on success or error description. | 359 360 361==== Example usage with Python ==== 362 363First enable [[devel:xmlrpc|DokuWiki's XML-RPC API]] as described. See also [[https://python-dokuwiki.readthedocs.io/en/latest/|python-dokuwiki]]. 364<code python>#pip install dokuwiki 365import dokuwiki 366wiki = dokuwiki.DokuWiki('https://path/to/dokuwiki', 'user', 'password') 367wiki.send("plugin.json.get", "path:to:page") 368 369person = {'firstName': 'James', 'lastName': 'Smith', 'age': 40} 370wiki.send("plugin.json.set", "path:to:page", "id_attr", person) 371</code> 372
