Evaluation framework – `run_all`¶

This section describes usage instructions for researchers wishing to use flitsr as an evaluation framework for SBFL-related research.

`run_all` command line arguments¶

usage: run_all [-h] [-m METRIC [METRIC ...]] [-M METRIC [METRIC ...]] [-i PAT]
               [-e PAT] [-d DEPTH] [-f [PAT] | -D [PAT] | -t [EXT] | -g]
               [-c CPUS] [-r] [-a FLITSR_ARG [FLITSR_ARG ...]] [-p DRIVER]
               [-C]

Named Arguments¶

-m, --metrics

Possible choices: ample, anderberg, arith_mean, barinel, cohen, dice, dstar, euclid, fleiss, geometric, goodman, gp13, hamann, hamming, harmonic, hyperbolic, jaccard, kulczynski1, kulczynski2, m1, m2, naish2, ochiai, ochiai2, overlap, rogers_tanimoto, rogot1, rogot2, russell_rao, sbi, scott, simpl_match, sokal, sorensen_dice, tarantula, wong1, wong2, wong3, zoltar, artemis, sbfl, flitsr, multi, parallel

Runs only the given metrics (can be specified multiple times)

-M, --exclude-metrics

Excludes the given metrics (can be specified multiple times)

-i, --include

Include directories matching PAT in run (can be specified multiple times)

-e, --exclude

Exclude directories matching PAT in run (can be specified multiple times)

-d, --depth

Specifies the depth at which to look for inputs (starting at 0 for the current directory)

-f, --file

Look only for single file inputs (that optionally match pattern PAT)

-D, --dir

Look only for directories as inputs (that optionally have a sub-file matching pattern PAT)

-t, --tcm

Look only for TCM type inputs (with optional extension EXT)

-g, --gzoltar

Look only for GZoltar type inputs

-c, --num-cpus

Sets the number of CPUs to run in parallel on (default automatic)

-r, --recover

Recover from a partial run_all run by re-using existing files

Default: False

-a, --flitsr_arg

Specify an argument to give to the flitsr program. NOTE: use -a=”<argument>” syntax for arguments beginning with a dash (“-“)

-p, --driver

Specify an alternate flitsr driver to use for running

See Creating custom drivers for a description of creating your own driver.

-C, --collect-results

Only collect already generated run files in the current directory into results file(s). Ignores most other options, except those for metrics (see -m and -M).

Default: False

Evaluation framework – run_all¶

run_all command line arguments¶

Named Arguments¶

Evaluation framework – `run_all`¶

`run_all` command line arguments¶