JULIUS(1)

NAME
Julius - open source multi-purpose LVCSR engine

SYNOPSIS
julius [-C jconffile] [options ...]

DESCRIPTION
Julius is a high-performance, multi-purpose, open source speech recognition engine that performs almost real-time recognition of continuous speech with 60k-word vocabulary on most current PCs.

Word 3-gram language model and triphone HMM acoustic model of any units and sizes can be used. As standard formats are adopted for the models, users can use their own language and acoustic models with Julius to build recognition system of their own.

Julius can perform recognition on audio files, live microphone input, network input and feature parameter files. The maximum size of vocabulary is 65,535 words.

RECOGNITION MODELS
Julius supports the following models. Sub-word HMM (Hidden Markov Model) in HTK format are supported. Phoneme models (monophone), context dependent phoneme models (triphone), tied-mixture and phonetic tied-mixture models of any unit can be used. When using context dependent models, interword context is also handled. The system uses 2-gram and reverse 3-gram language models. The Standard ARPA format is supported. In addition, a binary format N-gram is also supported for efficiency. The binary N-gram can be converted from the ARPA language models using the attached tool mkbingram.

SPEECH INPUT
Speech waveform files (16bit WAV (no compression), RAW format, and many other if used with libsndfile library) and feature parameter files (HTK format) can be used as speech input. Live input from either a Microphone, a DatLink (NetAudio) system, or via tcpip network is also supported.

Notice: Julius can only extract MFCC_E_D_N_Z features internally. If you want to use HMMs based on another type of feature extraction then microphone input and speech waveform files cannot be used. Use an external tool such as Hcopy or wav2mfcc to create the appropriate feature parameter files.

SEARCH ALGORITHM
Recognition algorithm of Julius is based on a two-pass strategy. Word 2-gram and reverse word 3-gram is used on the respective passes. The entire input is processed on the first pass, and again the final searching process is performed again for the input, using the result of the first pass as a "guidance". Specifically, the recognition algorithm is based on a tree-trellis heuristic search combined with left-to-right frame-synchronous beam search and right-to-left stack decoding search.

When using context dependent phones (triphones), interword contexts are taken into consideration. For tied-mixture and phonetic tied-mixture models, high-speed acoustic likelihood calculation is possible using gaussian pruning.

For more details, see the related document or web site below.

OPTIONS
The options below allow you to specify the models and set system parameters. You can set these option at the command line, however it is recommended that you combine these options in a "jconf settings file" and use the "-C" option to read it at run time.

Below is an explanation of all the available options.

  Speech Input
Select speech data input source. 'rawfile' is from waveform file (file name should be specified after startup). 'mfcfile' is a feature vector file extracted by HTK HCopy tool. 'mic' means live microphone input, and 'adinnet' means receiving waveform data via tcpip network from an adinnet client. 'stdin' means standard tty input.

The supported waveform file format varies based on compilation time configuration. To see what format is actually supported, see the help message using option "-help". (for stdin input, only WAV (no compression) and RAW (16bit, BE) is supported.)
(default: mfcfile) (with -input rawfile|mfcfile) perform recognition on all files contained within the target file. (with -input adinnet) adinnet port number (default: 5530) (with -input netaudio) set the server name and unit ID of the Datlink unit. auto-save recognized speech data under the directory. Each segmented inputs are recorded each by one, with a filename of "YYYY.MMDD.HHMMSS.raw", which shows the system time when the input begins (YYYY=year, MMDD=month/day, HHMMSS=hour/minute/second). The file format is RAW, 16bit, 16kHz, mono, big endian.

  Speech Detection
Force silence cutting (=speech segment detection) ON/OFF. (default: ON for mic/adinnet, OFF for files) Amplitude threshold (0 - 32767). If the amplitude passes this threshold it is considered to be the beginning of a speech segment, if it drops below this level then it is the end of the speech segment. (default: 3000) Zero crossing threshold per a second (default: 60) Margin at the start of the speech segment in milliseconds. (default: 300) Margin at the end of the speech segment in milliseconds. (default: 400) On some sound devices, invalid "0" samples may be recorded at the start and end of recording. Julius remove them automatically by default. This option inhibit the automatic removal.

  Acoustic Analysis
Sampling frequency (Hz).
(default: 16kHz = 625ns). Sampling rate (nanoseconds).
(default: 625ns = 16kHz). Analysis window size (No. samples) (default: 400). Frame shift (No. samples) (default: 160). Delta window size (No. frames) (default: 2). High-pass filter cutoff frequency (Hz).
(default: -1 = disabled) Low-pass filter cutoff frequency (Hz).
(default: -1 = disabled) Perform spectral subtraction using the head silence of files. Valid only for rawfile input. Specify the length of head silence in milliseconds (default: 300) Perform spectral subtraction for speech input using pre-estimated noise spectrum from file. The noise spectrum data should be computed beforehand by mkss. Alpha coefficient of spectral subtraction. Noise will be subtracted stronger as this value gets larger, but distortion of the resulting signal also becomes remarkable. (default: 2.0) Flooring coefficient of spectral subtraction. For spectral parameters that go under zero after subtraction, the source signal is assigned with this coefficient multiplied. (default: 0.5)

  Language Model (word N-gram)
2-gram language model filename in standard ARPA format. Reverse 3-gram language model filename. This is required for the second search pass. If this is not defined then only the first pass will take place. Use a binary language model as built using mkbingram(1). This is used in place of the "-nlr" and "-nlr" options above, and allows Julius to perform rapid initialization. Language model score weights and word insertion penalties for the first and second passes respectively.

The hypothesis language scores are scaled as shown below:

lm_score1 = lm_weight * 2-gram_score + lm_penalty lm_score2 = lm_weight2 * 3-gram_score + lm_penalty2

The defaults are dependent on acoustic model:

  First-Pass | Second-Pass
 --------------------------
  5.0 -1.0 | 6.0 0.0 (monophone)
  8.0 -2.0 | 8.0 -2.0 (triphone,PTM)
  9.0 8.0 | 11.0 -2.0 (triphone,PTM, setup=v2.1) Additional insertion penalty for transparent words. (default: 0.0)

  Word Dictionary
Word dictionary file (required) Sentence start and end silence word as defined in the dictionary. (default: "<s>" / "</s>")

These are dealt with specially during recognition to hypotheses start and end points (margins). They can be defined as shown below.

     (Word_ID is the word position in the dictionary
      file starting from 0) Disregard dictionary errors. Word definitions with errors will be skipped on startup.

  Acoustic Model (HMM)
HMM definition file to use. (required) HMMList file to use. Required when using triphone based HMMs. This file provides a mapping between the logical triphones names genertated from the phonetic representation in the dictionary and the HMM definition names. When using a triphone model, select method to handle inter-word triphone context on the first and last phone of a word in the first pass.

max: use maximum likelihood of the same
     context triphones (default)
avg: use average likelihood of the same
     context triphones Normally Julius determines whether the specified hmmdefs is a context-dependent model by the model definition names, i.e., whether the model names contain character '+' and '-'. In case the automatic detection fails, you can explicitly specify by these options. These options will override the automatic detection result. Disable check of the input parameter type. (default: enabled)

  Acoustic Computation
Gaussian Pruning will be automatically enabled when using tied-mixture based acoutic model. Gaussian Selection needs a monophone model converted by mkgshmm to activate. With Gaussian Pruning, specify the number of Gaussians to compute per codebook. (default: 2) Set the Gaussian pruning technique to use.
(default: safe (setup=standard) beam (setup=fast)) Specify monophone hmmdefs to use for Gaussian Mixture Selectio. Monophone model for GMS is generated from an ordinary monophone HMM model using mkgshmm. This option is disabled by default. (no GMS applied) When using GMS, specify number of monophone state to select from whole monophone states. (default: 24)

  Inter-word Short Pause Handling
Add a word entry to the dictionary that corresponds to inter-word short pauses. The content of the word entry can be specified by "-iwspentry". Specify the word entry that will be added by "-iwspword". (default: "<UNK> [sp] sp sp") (Multi-path version only) Enable inter-word context-free short pause handling. This option appends a skippable short pause model for every word end. The added model will also be ignored in context modeling. The model to be appended can be specified by "-spmodel" option. Specify short-pause model name that will be used in "-iwsp". (default: "sp")

  Short-pause Segmentation
The short pause segmentation can be used for sucessive decoding of a long utterance. Enabled when compiled with '--enable-sp-segment'. Set the short-pause duration threshold in number of frames. If a short-pause word has the maximum likelihood in successive frames longer than this value, then interrupt the first pass and start the second pass. (default: 10)

  Search Parameters (First Pass)
Beam width (Number of HMM nodes). As this value increases the precision also increases, however, processing time and memory usage also increase.

default value: acoustic model dependent
  400 (monophone)
  800 (triphone,PTM)
 1000 (triphone,PTM, setup=v2.1) Number of high frequency words to be separated from the lexicon tree. (default: 150) Only perform the first pass search. This mode is automatically set when no 3-gram language model has been specified (-nlr). Explicitly specify whether real-time (pipeline) processing will be done in the first pass or not. For file input, the default is OFF (-norealtime), for microphone, adinnet and NetAudio input, the default is ON (-realtime). This option relates to the way CMN is performed: when OFF CMN is calculated for each input independently, when the realtime option is ON the previous 5 second of input is always used. Also refer to -progout. Save last CMN parameters computed while recognition to the specified file. The parameters will be saved to the file in each time a input is recognized, so the output file always keeps the last CMN parameters. If output file already exist, it will be overridden. Load initial CMN parameters previously saved in a file by "-cmnsave". This option enables Julius to recognize the first utterance of a live microphone input or adinnet input with CMN.

  Search Parameters (Second Pass)
Beam width (number of hypothesis) in second pass. If the count of word expantion at a certain length of hypothesis reaches this limit while search, shorter hypotheses are not expanded further. This prevents search to fall in breadth-first-like status stacking on the same position, and improve search failure. (default: 30) The search continues till 'candidate_num' sentence hypotheses have been found. The obtained sentence hypotheses are sorted by score, and final result is displayed in the order (see also the "-output" option).

The possibility that the optimum hypothesis is found increases as this value is increased, but the processing time also becomes longer.

Default value depends on the engine setup on compilation time:
  10 (standard)
   1 (fast, v2.1) The top N sentence hypothesis will be Output at the end of search. Use with "-n" option. (default: 1) Score envelope width for enveloped scoring. When calculating hypothesis score for each generated hypothesis, its trellis expansion and viterbi operation will be pruned in the middle of the speech if score on a frame goes under [current maximum score of the frame- width]. Giving small value makes computation cost of the second pass smaller, but computation error may occur. (default: 80.0) The maximum number of hypothesis that can be stored on the stack during the search. A larger value may give more stable results, but increases the amount of memory required. (default: 500) Number of expanded hypotheses required to discontinue the search. If the number of expanded hypotheses is greater then this threshold then, the search is discontinued at that point. The larger this value is, the longer the search will continue, but processing time for search failures will also increase. (default: 2000) When performing word expansion, this option sets the number of frames before and after in which to determine next word hypotheses. This prevents the omission of short words but, with a large value, the number of expanded hypotheses increases and system becomes slow. (default: 5)

  Forced Alignment
Do viterbi alignment per word units from the recognition result. The word boundary frames and the average acoustic scores per frame are calculated. Do viterbi alignment per phoneme (model) units from the recognition result. The phoneme boundary frames and the average acoustic scores per frame are calculated. Do viterbi alignment per HMM state from the recognition result. The state boundary frames and the average acoustic scores per frame are calculated.

  Server Module Mode
Run Julius on "Server Module Mode". After startup, Julius waits for tcp/ip connection from client. Once connection is established, Julius start communication with the client to process incoming commands from the client, or to output recognition results, input trigger information and other system status to the client. The multi-grammar mode is only supported at this Server Module Mode. The default port number is 10500. (Only for Server Module Mode) Switch which symbols of recognized words to be sent to client. Specify 'W' for output symbol, 'L' for grammar entry, 'P' for phoneme sequence, 'S' for score, respectively. Capital letters are for the second pass (final result), and small letters are for results of the first pass. For example, if you want to send only the output symbols and phone sequences as a recognition result to a client, specify "-outcode WP".

  Message Output
Output the language and acoustic scores separately. Omit phoneme sequence and score, only output the best word sequence hypothesis. Enable progressive output of the partial results on the first pass at regular intervals. set the output time interval of "-progout" in milliseconds. Equivalent to "-progout -quiet"

  OTHERS
(For debug) display internal status and debug information. Load the jconf file. The options written in the file are included and expanded at the point. This option can also be used within other jconf file. (For debug) turn on interactive check mode of tree lexicon structure at startup. (For debug) turn on interactive check mode of model mapping between Acoustic model, HMMList and dictionary at startup. Display version information and exit. Display a brief description of all options.

EXAMPLES
For examples of system usage, refer to the tutorial section in the Julius documents.

NOTICE
Note about path names in jconf files: relative paths in a jconf file are interpreted as relative to the jconf file itself, not to the current directory.

SEE ALSO
julian(1), mkbingram(1), mkss(1), jcontrol(1), adinrec(1), adintool(1), mkdfa(1), mkgsmm(1), wav2mfcc(1)

http://julius.sourceforge.jp/ (main)
http://sourceforge.jp/projects/julius/ (development site)

DIAGNOSTICS
Julius normally will return the exit status 0. If an error occurs, Julius exits abnormally with exit status 1. If an input file cannot be found or cannot be loaded for some reason then Julius will skip processing for that file.

BUGS
There are some restrictions to the type and size of the models Julius can use. For a detailed explanation refer to the Julius documentation. For bug-reports, inquires and comments please contact julius@kuis.kyoto-u.ac.jp or julius@is.aist-nara.ac.jp.

AUTHORS
Designed by Tatsuya KAWAHARA and Akinobu LEE (Kyoto University)

Development by Akinobu LEE (Kyoto University) Development of above versions by Akinobu LEE (Kyoto University) Development of above versions by Akinobu LEE (Nara Institute of Science and Technology)

THANKS TO
From Rev.3.2 Julius is released by the "Information Processing Society, Continuous Speech Consortium".

The Windows DLL version was developed and released by Hideki BANNO (Nagoya University).

The Windows Microsoft Speech API compatible version was developed by Takashi SUMIYOSHI (Kyoto University).