[PF:172765] Only 10 Words Make Up 25% Of The English Language

Only 10 Words Make Up 25% Of The English Language

 

 

One of every four words you say or write is probably one of these: "the," "be," "to," "of," "and," "a," "in," "that," "have," and "I."

Those ten words, listed in order of frequency, comprise around 25% of the recorded English language, according to an ambitious project at Oxford University.

The project, called the Oxford English Corpus, is a growing database of examples from 21st-century English, ranging from literature and scientific journals to emails. The Corpus contains more than two billion instances of words, called "tokens."

"A type is a unique string of letters, regardless of how often it is used.  A token is a single occurrence of a type. The sentence 'the cat sat on the mat' contains six tokens but five types, because there are two occurrences of the type 'the,'" Professor Patrick Hanks, former editor of English dictionaries at the Oxford University Press, told Business Insider. 

The Corpus hit two billion tokens in 2010. Lexicographers then deduced the ten words that appear the most. But a Harvard professor named George Kingsley Zipf had already predicted the result back in 1935.

 "The weak version of Zipf's Law says that words are not evenly distributed across texts; instead, there are a few words that are very common and a very large number of words that are very rare. And there is a neat curve linking the two extremes. Useful words such as 'useful' and 'curve' are quite low on the curve; boring words like 'thing,' 'go,' 'say,' 'give' and 'take' are quite high on the curve," Hanks said.

Hanks doesn't mean "neat" as a outdated form of "cool," either. He means orderly, organized, statistically beautiful.

The ten aforementioned words comprise about 25% of our language. Going further, the top 100 words comprise about 50% of our language, while 50,000 words comprise 95% of our language. To account for the last 5%, we need a vocabulary of more than a million words.

To test the theory, I counted the number of times each of the ten words appears in this article — 98 out of 391. Thus, "the," "be," "to," "of," "and," "a," "in," "that," "have," and "I" make up about 25.06% of this article. Right on the money.

If we consider "content words" (words with tangible meaning) instead of "function words," the top ten list changes to include: "time," "person," "year," "way," "day," "thing," "man," "world," "life," and "hand."

 

 

 

--

Good programming is 99% sweat and 1% coffee.
http://rehansheik.blogspot.com
If you forward this email, please delete the forward history, including my email address. Remember, erasing the history helps to prevent SPAMMERS from  mining addresses and viruses from being propagated.

--
--
From:
[Pak-Friends] Group Member
Visit Group: http://groups.google.com/group/Karachi-786
Subscription: http://groups.google.com/group/karachi-786/subscribe
===========================================================
¸,.-~*'¨¯¨'*·~-.¸¸,.-~*'[PäK¤.¸.¤F®ï£ñD§]'*·~-.¸¸,.-~*'¨¯¨'*·~-.¸
===========================================================
All members are expected to follow these Simple Rules:
-~----------~----~----~----~------~----~------~--~---
Be Careful in Islamic Discussions;
Bad language and insolence against Prophets (and / or their companions, Islamic Scholars, and saints) is an Instant ban.
Abuse of any kind (to the Group, or it's Members) shall not be tolerated.
SPAM, Advertisement, and Adult messages are NOT allowed.
This is not Dating / Love Group, avoid sending personnel messages to group members.
---
You received this message because you are subscribed to the Google Groups "Pak Friends" group.
To unsubscribe from this group and stop receiving emails from it, send an email to karachi-786+unsubscribe@googlegroups.com.
To post to this group, send email to karachi-786@googlegroups.com.
Visit this group at http://groups.google.com/group/karachi-786.
For more options, visit https://groups.google.com/groups/opt_out.