University of Hertfordshire

Documents

  • C. Lyon
  • C. Matthews
View graph of relations
Original languageEnglish
PublisherUniversity of Hertfordshire
Publication statusPublished - 1995

Publication series

NameUH Computer Science Technical Report
PublisherUniversity of Hertfordshire
Volume212

Abstract

There are statistical patterns of letter sequences in natural language, and different languages have different characteristic patterns. This effect can be used to determine in which language a text is written. The patterns are captured with a single layer, feed forward neural net trained in supervised mode. The sequential dependencies of letters are modelled by taking adjacent letter pairs and letter triples. Training and test data are converted to sets of these tuples, which are the basic elements classified by the network. This approach is supported by information theoretic results on the entropy of letter sequences for English. The architecture of the network used is shown to be appropriate for data with the characteristics of natural language letter sequences. For 3 languages over 99% of test strings are correct. For 4 languages, including Dutch and German which are similar, over 92% are correct.

ID: 89800