Arcane tribal languages in remote settings (South America, Asia, Africa) would be the most daunting for English speakers. This is because of their isolation from the mainstream languages of more populated areas, and therefore have little in common with familiar Roman and Greek roots in the Indo-European tradition.
If you want to try your hand at a south/eastern European language, try Romanian. It is the only Latin-based language in that geography, and shows many cognate commonalities with English. And although Danish is a close cognate to English, its 27 phonetically distinct vowels make it much harder to understand and master than Swedish and Norwegian. Also complicating Danish are its varied glottal stops that are both hard to hear and hard to pronounce for non-native speakers. While Danes can pick up both these Nordic systems, both the two other speaking groups have more trouble with Danish, and pronunciation is exacting (Jens Lund, Ph.D., folklorist and native Danish speaker).
Are you up for a real language challenge that will allow you to speak to under 100 other specialists after years of work? Then consider the constructed language (conlang) domain. Klingon would have to be one of the most challenging. This language was invented for Star Trek III (1984) as a formal integrated speech for the Klingons in the Trek universe. After a dictionary was published, many people dabbled in its difficult spelling and pronunciation, but only a handful (under 100 estimated) have become skilled speakers able to converse with each other and understand the film tracks. In addition, since Klingon speech focuses largely on spacecraft and warfare, it has limited use for day-to-day conversation. It is popular with linguists for its creative aspects played out within the general principles of language.
Of natural languages (as opposed to constructed cases), it is interesting that Mandarin Chinese is the hardest to learn for English speakers—because of the thousands of ideographs necessary for written comprehension, as well as a four-tone scale for meaning. But it is also the most widely spoken global language (besides Globish, basic English spoken as an auxiliary tongue). Arabic, Polish, and Russian follow, the first also forcing a totally unfamiliar writing system. The US Foreign Service Institute groups languages for difficulty from 1 through 4, with the “Super-hard” Category 4 including Arabic, Chinese (Cantonese and Mandarin), Japanese, and Korean.
Classical languages are more difficult simply because of their restricted lives in religious and academic contexts, but express a range. "Classical Greek and Sanskrit are extremely difficult because they are so highly inflected--hundreds of forms of the verb and numerous case endings. Late Greek (koine) simplifies the grammar and thus is much easier to read and not particularly difficult. Egyptian grammar and vocabulary are very simple. Its only real difficulty is mastering the hieroglyphs, which are very few compared to Chinese" (Prof. Robert Littman, Classical studies, University of Hawaii at Manoa).
How about learning a tribal language? There are many still active around the globe, the most in New Guinea (numbering around 850), the most diverse linguistic area known. The Khoisan language of South Africa is among the world’s oldest, at 60,000 years. Closer, in the US, the three leading tribal languages still spoken are Navajo (by far the largest and hardest) in Arizona, Yupik in Central Alaska, and Sioux in the upper Midwest and Canada. Navajo was famously employed as an unbreakable talking code by native-speaking marines in WWII. They are all difficult, made more so by their roots in exotic and ancient cultures, arcane to learn and relate to vocabulary—and have only in modern times enjoyed a written format. Hawaii is the only US state with two official languages—Hawaiian and English--as of 1978. Along the range of tribal language difficulty, Hawaiian is among the easiest.
So there are many “hardest languages” out there to appreciate, if not to master as a fluent speaker, and each has a rich cultural component. Klingon was born from the constructed science fiction of the Star Trek universe, so does have a soundtrack, but a steep learning for pronunciation, structure, and symbol alphabet. Klingon was designed to look and sound truly alien, which it does as a function of its weirdly off-center profile without cognates. (However, Duolingo now actually offers the course.) The most widely spoken constructed lingua franca, Esperanto, has an estimated million speakers world-wide, but little cultural baggage (literature, history, religion, film, cuisine). However, fluency can be reached in one-tenth the time of natural languages, and in itself, Esperanto offers a quickly effective base for language-learning capability.
Next blog: “Easy” Languages