This tutorial presents a number of example scripts which will show how to use the realtime module in practice. We move from simple to more complex examples. All the examples make use of the ft_realtime_asynchronous or ft_realtime_synchronous functions, which process incoming data either continuously or trigger-based using a specified bcifun. The bcifun performs an analysis on individual trials (e.g., preprocessing, feature extraction, and classification).

In this example, we will use an offline MEG dataset and show how data can be processed asynchronously. We use a dataset where the subject was required to covertly attend to the left and right visual field and we aim to compute the alpha lateralization index (ALI) online. For online simulation using offline data we need to specify the dataset:

cfg.dataset = '~/data/alphalat/Sander_AlphaLat_20080530_01.ds'; % path to our dataset

Also, we need to specify that we want to start reading data from the beginning of the file:

cfg.jumptoeof  = 'no';
cfg.bufferdata = 'first';

In online applications we would use jumptoeof='yes' and bufferdata='last' to ensure that we are always reading the most recent data segment.

As our bcifun we use the bcifun_latidx function, which computes a lateralization index between left and right hemisphere occipital alpha power:

cfg.bcifun = @bcifun_latidx

Now we may start the simulation as follows:

ft_realtime_asynchronous(cfg);

The output of this simple script is something like the following:

>> ft_realtime_asynchronous(cfg);
processing segment 1 from sample 1 to 1200
generated command 1
processing segment 3 from sample 1201 to 2400
generated command 2
processing segment 5 from sample 2401 to 3600
generated command 2
processing segment 7 from sample 3601 to 4800
generated command 2
processing segment 9 from sample 4801 to 6000
generated command 0

Hence, it will print the output of the bcifun, namely the computed ALI. Typically, we wouldn't want to print the output but rather send it to some other application that is able to feed the command back to the user. This is realized by the ''cfg.ostream' option which will specify how the command is send via ft_write_event.

Asynchronous BCI proceed similarly but processed data trigger-based instead of continuously. This is specified as follows:

cfg.trigger = [4 2]; % left (2) and right (4) covert attention

Combining this with the previous fields, we will only process the segments of interest:

>> ft_realtime_synchronous(cfg);
found 2578 events
processing segment 1 from sample 30965 to 32165, trigger = 2
generated command 2
processing segment 3 from sample 30965 to 32165, trigger = 2
generated command 2
processing segment 5 from sample 41770 to 42970, trigger = 4
generated command 0
processing segment 7 from sample 41770 to 42970, trigger = 4
generated command 0
processing segment 9 from sample 47172 to 48372, trigger = 4
generated command 0
processing segment 11 from sample 47172 to 48372, trigger = 4

In the above examples, we have shown how to compute the ALI using asynchronous or synchronous realtime processing. In a BCI setup, we typically have a training session in which data is collected to build a classifier and a test session in which the BCI is evaluated.

It is useful to make a distinction between between the acquisition machine (where neuroimaging data arrives), the analysis machine (where the data is analyzed) and the presentation machine (where visual stimulation is provided). In EEG setups these machines are often one and the same. In MEG or fMRI setups all three machines may be different physical machines.

During training, the analysis machine typically receives data from the acquisition machine together with associated class labels (i.e., it is trigger-based). Based on this incoming data, the bcifun will build up a classifier either by storing the input data or training a classifier online. Assuming the former, we may do something like

cfg.nsamples = 100;
cfg.bcifun = @my_train_fun;
out = ft_realtime_synchronous(cfg)

cfg.clf = ft_mv_analysis({ft_mv_standardizer ft_mv_svm});
cfg.clf = clf.train(out{1},out{2});

cfg.bcifun = @my_test_fun;
ft_realtime_asynchronous(cfg)

Here, cfg.nsamples specifies how many samples should be processed and out contains the (preprocessed) input data X and a vector of class labels Y. These then act as input to a multivariate analysis (see the multivariate module). The trained analysis is then passed to an asynchronous setup which processed each data segment using the specified classifier.