History: Letter Frequencies

Source of version: 2 (current)

I've just generated new letter frequency data based on all but the
first section of:

[http://www.teddyb.org/~rlpowell/hobbies/lojban/grammar/test_sentences.txt|test_sentences.txt]

So basically, the CLL, Alice, and a bunch of IRC.  If people would
like to suggest other non-trivially sized Lojban texts to add,
please let me know, but we've got ~650K characters here, so I think
the statistics is pretty good.

My data, sorted by number of occurences:

  85004 i
  68959 a
  52225 e
  50517 u
  47944 o
  43807 l
  36358 n
  33169 c
  27097 m
  24514 r
  22989 s
  21356 d
  20536 '
  18317 t
  17749 k
  14459 b
  13359 p
  11990 j
   8810 g
   8007 z
   6857 v
   6616 x
   6288 f
   4580 y


As ratios:

0.130472888242183 i
0.105845370809523 a
0.080160305261493 e
0.077538691065483 u
0.073589385839292 o
0.067239492438300 l
0.055806000549495 n
0.050911195121464 c
0.041591264560472 m
0.037626610305031 r
0.035285883344307 s
0.032779386867677 d
0.031520766469124 '
0.028114816878406 t
0.027242992016969 k
0.022193161393507 b
0.020504768175936 p
0.018403486071523 j
0.013522494769818 g
0.012289967720991 z
0.010524829357167 v
0.010154917752226 x
0.009651469592805 f
0.007029855396795 y

The only previous work on this I'm aware of is:

[http://www.lojban.org/files/papers/scrabble.unf|The Scrabble Paper]

Which, it turns out, is amazingly flawed (which is fine, because
that was a long time ago!).

Using the data without lujvo, we have:

i       1045
a       991
u       642
n       563
e       496
r       460
o       395
t       361
c       360
l       348
s       339
'       316
k       285
m       254
j       249
d       219
b       212
p       203
f       149
g       146
v       119
x       108
z       87
y       19

which is only marginally different from what I have.

Using the data with lujvo, however, which IIRC is what the Scrabble
frequencies were based on, we have the obviously biased:

y     5553
r     2979
a     2949
i     2678
n     2047
u     1755
e     1560
l     1395
s     1363
t     1359
k     1107
m     1048
o     1046
c     1040
'     1012
j     1008
p     872
b     865
d     862
f     616
g     589
x     532
v     490
z     359

-((Robin Lee Powell))
History: Letter Frequencies

Source of version: 2 (current)

Search Lojban Resources
