pyeeg.io.load_wordfreq_values

pyeeg.io.load_wordfreq_values(filepath, key='frequency', unkval=111773390, normfactor=3213700000000.0)

Load word frequency, and returns -log of it (scaled by median value).

Parameters:

• filepath (str) – Path to the file containing word counts, e.g. as extracted from Google Unigrams. If None, returns None.

• key (str) – Name of the column containing the word counts (Default: ‘frequency’).

• unkval (int) – Value used to replace unknown word counts (i.e. entries equal to -1) before transformation (Default: 111773390, global median from stories).

• normfactor (float) – Normalization factor, roughly equal to the total word count, used to turn raw counts into probabilities (Default: 3.2137e12).

Returns:

wordfreq – Negative log frequency (i.e. log probability) of each word.

Return type:

ndarray