144 lines
6.4 KiB
Text
144 lines
6.4 KiB
Text
newfol Manual
|
|
=============
|
|
:doctype: book
|
|
|
|
== Using newfol
|
|
|
|
=== Overview
|
|
|
|
newfol is a combination user interface and database store for small databases
|
|
that can fit into memory. It is designed to represent textual data (stored and
|
|
rendered as UTF-8) and provides tools to search this data using Python regular
|
|
expressions.
|
|
|
|
newfol is not inherently transactional, like some other databases; however, it
|
|
provides optional support for storing each saved state into Git to preserve a
|
|
complete history of changes. Along with an SHA-2 hash, this provides immediate
|
|
detection of and recovery from potential data loss situations.
|
|
|
|
== Data Layout
|
|
|
|
Each newfol database is stored in a directory, which can be specified with the
|
|
`--homedir` option. The default is `~/.newfol`.
|
|
|
|
Each newfol database contains a `schema` file, which describes the layout of the
|
|
data, the methods for importing and exporting it, and other metadata about the
|
|
intended behavior of the database. Refer to <<_schema_file>> for a description
|
|
of the contents of this file. It is the only file which may be hand-edited.
|
|
|
|
Each database also contains a `dtb` file, which contains the actual data. This
|
|
will be in CSV (colon-separated values), pickle, or JSON format, depending on
|
|
the contents of the `version` file. It may also be compressed.
|
|
|
|
The `dtb.checksum` file provides a cryptographic SHA-2 hash and length to ensure
|
|
that the data has not been accidentally corrupted. newfol will not open a
|
|
database where the database contents do not match this file.
|
|
|
|
The `lock` file is used to prevent access by multiple processes, which may
|
|
accidentally corrupt each other.
|
|
|
|
=== Schema File
|
|
|
|
The schema file is a CSV (colon-separated values) file. The first entry in the
|
|
file consist of the values `fmt`, the format version number, and `newfol schema
|
|
file`.
|
|
|
|
newfol automatically upgrades schema files to the latest version, which at the
|
|
time of this writing, was 3. If no schema file exists, newfol will
|
|
automatically create a version 0 schema file and upgrade it to the latest
|
|
version on start. newfol does not accept schema files that have a newer version
|
|
than it currently supports, but it ignores directives it does not understand,
|
|
and so can be manually downgraded by editing the `fmt` directive.
|
|
|
|
Any record beginning with a `#` is ignored as a comment and preserved on
|
|
upgrades. Extra fields in a record are ignored.
|
|
|
|
The standard rules for formatting colon-separated values apply. In other words,
|
|
the format is defined as identical to the `text/csv` format specified in RFC
|
|
4180, except that the delimiter is a colon instead of a comma and that the line
|
|
break is a single linefeed instead of a CRLF pair.
|
|
|
|
==== Format Directives (`fmt`)
|
|
|
|
Format directives define the file as a schema file or configuration file as
|
|
specified in <<_schema_file>> and <<_configuration_file>>. They must appear
|
|
only at the very beginning of a schema or configuration file. In particular, it
|
|
is not allowed for them to be preceded by a byte order mark.
|
|
|
|
==== Table Definition Directives (`def`)
|
|
|
|
A table definition directive consists of a record containing the values `def`
|
|
and the table name. A table must be defined before any field definitions can
|
|
occur for that table.
|
|
|
|
==== Field Definition Directives (`fld` and `dsc`)
|
|
|
|
Field definitions come in two types. The first, `fld`, provides a mapping for
|
|
imported data, and `dsc` does not. They are otherwise identical.
|
|
|
|
A field definition directive consists of a record containing the values `fld` or
|
|
`dsc`, the name of the table, the field number for imported data (or empty for
|
|
`dsc`), the field number for use in newfol, an optional Python 3 expression, and
|
|
a description.
|
|
|
|
The Python 3 expression is evaluated and placed in the field when creating a new
|
|
record. This can be used, for example, to automatically insert the date into a
|
|
particular field. This requires that code execution has been allowed by an
|
|
`exe` directive.
|
|
|
|
==== Execution Directives (`exe` and `execute`)
|
|
|
|
Execution directives specify whether Python code will be run from the schema
|
|
file. Such a directive consists of a record containing either `exe` or
|
|
`execute` and then either `yes` or `no`.
|
|
|
|
Unlike other directives, where the schema file overrides the configuration file,
|
|
if any execution directive is set to `no`, execution is disabled. This is a
|
|
security feature to allow reading an untrusted database. Execution is also
|
|
disabled if more than one execution directive is seen in either the schema or
|
|
configuration file.
|
|
|
|
==== Key Field Directives (`key`)
|
|
|
|
Key field directives indicate the fields to display in the _Browse All Records_
|
|
view. By default, this value is 0 (display the first field).
|
|
|
|
==== Column Directives (`col`)
|
|
|
|
Column directives indicate how many columns to use to display fields. A column
|
|
directive consists of a record with the values `col` and an integer number of
|
|
columns.
|
|
|
|
==== Hook Directives (`txn` and `hook`)
|
|
|
|
Hook directives indicate a series of hooks to be run on each load or save
|
|
operation. Several different hooks are available. The default set of hooks is
|
|
`hash`. This provides an SHA-2 hash of the data in the `dtb.checksum` file, and
|
|
cannot be disabled, only replaced with a specific algorithm.
|
|
|
|
By default, the hash is `sha256` on 32-bit systems and `sha512` on 64-bit
|
|
systems; `sha384` is also available. If a specific algorithm is specified, it
|
|
overrides the `hash` default. Older versions of newfol only supported the
|
|
`sha256` algorithm.
|
|
|
|
The other available hook is `git`. This hook performs a commit every time the
|
|
file is saved, and a git checkout of the `dtb` file before every load. This
|
|
allows a user to keep a history of the data and to easily back it up to another
|
|
location.
|
|
|
|
A hook directive consists of a record starting with the value `txn`, and
|
|
followed by a list of hook names. `hook` may be used as a synonym for `txn`.
|
|
|
|
=== Configuration File
|
|
|
|
Configuration files are identical to schema files except that instead of the
|
|
text `newfol schema file` in the `fmt` entry, the text `newfol config file` is
|
|
substituted. Because newfol parses both files in the same way, any entry that
|
|
is valid in a schema file is also valid in a configuration file. That said,
|
|
some directives make more sense in a configuration file than others.
|
|
|
|
Common directives that are useful in a configuration file are transaction
|
|
settings, colors, and keybindings.
|
|
|
|
The default location is +_$XDG_CONFIG_HOME_/newfol/defaults+. If
|
|
`$XDG_CONFIG_HOME` is not defined, it defaults to +_$HOME_/.config+.
|