Getting started
Installation
Before we can do anything, we need a Python 3.7+ distribution. We recommend Miniconda. If you don’t already have it, go ahead and install it.
If you intend to install from source, you will also need Git.
Now that the prerequisites are satisfied, the next order of business is to install Timeflux and its dependencies. To keep things nice and clean, we will do this in a new virtual environment. Depending on your system, open a shell or command prompt:
conda create --name timeflux python=3.9 pytables bottleneck
conda activate timeflux
pip install timeflux
pip install timeflux_example
If everything went well, Timeflux is now installed. Hooray!
Basic usage
Applications are self-described in YAML files. Running an app is easy:
timeflux app.yaml
Let’s try!
First, download a very simple app that we will use as an example:
curl -O https://raw.githubusercontent.com/timeflux/timeflux/master/examples/test.yaml
If the timeflux environment is not already activated, do it:
conda activate timeflux
You can now run the test app:
timeflux -d test.yaml
You should see and a bunch of random numbers every second. Hit Ctrl+C to stop.
Did you notice the -d
flag in the command line? It’s a shorthand for --debug
, and this what allowed us to see the messages in the console.
Command line options
There are only a few options, and you can list them with:
timeflux --help
This should print:
usage: timeflux [-h] [-v] [-d] [-E ENV_FILE] [-e ENV] app
positional arguments:
app path to the YAML or JSON application file
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-d, --debug enable debug messages
-E ENV_FILE, --env-file ENV_FILE
path to an environment file
-e ENV, --env ENV environment variables
Besides the -d
flag we already discussed, two options are worth mentioning: -E
or --env-file
and -e
or --env
.
Environment
Storing an app configuration in the environment is a good practice. There are a few ways of doing this:
If a file named .env is found in the current directory or in any of its parent directories, it will be loaded. A .env file looks like this:
# A comment that will be ignored
FOO=bar
MEANING_OF_LIFE=42
As we saw earlier, you can also specify a custom path to an environment file with the --env-file
option.
Another way of setting environment variables is with the -e
option:
timeflux -e FOO="bar" -e MEANING_OF_LIFE=42 app.yaml
Finally, you can temporarily set environment variables for the duration of the session, directly from the console.
Windows:
set FOO "bar"
Linux, MacOS:
export FOO="bar"
The following environment variables are understood by Timeflux:
TIMEFLUX_LOG_LEVEL_CONSOLE
– This is the level of details printed in the console. Possible values are DEBUG, INFO, WARNING, ERROR and CRITICAL. The default value is INFO. Running thetimeflux
command with the-d
flag is the same as setting this variable to DEBUG.TIMEFLUX_LOG_LEVEL_FILE
– This is the logging level when the output of the application is written to a file. This variable accepts the same values as previously. The default value isDEBUG
.TIMEFLUX_LOG_FILE
– If set to a valid path, Timeflux will write the application output to a log file. Standard format codes are accepted.TIMEFLUX_SLEEP
– When a graph has a rate of zero, it will run as fast as possible, but will result in a high CPU load. Setting this variable to a non-zero value can help mitigating this issue. Default is 0.TIMEFLUX_HOOK_PRE
– Name of a Python module that will be run before executing the app.TIMEFLUX_HOOK_POST
– Name of a Python module that will be run after executing the app.
Others variables may be used by specific nodes and plugins. Refer to the relevant documentation for details.
By combining environment variables and templating, you can add logic to your YAML files and build configurable applications.
Plugins
Timeflux is modular. The timeflux
Python package contains the core features and the most essential nodes. Plugins are standard Python packages that provide one or several nodes. Officially supported plugins can be found on Timeflux GitHub page. Some plugins (especially those dealing with hardware) have special requirements. Please refer to each plugin repository for installation instructions.
Notable plugins include: