The Magic Mountain - Thomas Mann (for a more music-oriented work of similar geniusl try Dr. Faustus). Also, try to skip the Harriet Lowe-Porter translation (good, but not as great as the more recent one by Woods).
Light In August - William Faulkner. Chapter 6, first sentence: Memory believes before knowing remembers. Take it from there.
The Amazing Adventures of Kavalier & Clay - Michael Chabon.
The German Lesson - Seigfried Lenz
Ubik - Philip K. Dick
Those are very different books, all novels, and all have changed my perceptions of life in a profound way that survives the test of time (so far). I have read each at least 2 times, seperated by decades, and found increased value with time
I would also add Sometimes A Great Notion by Ken Kesey, Huckleberry Finn by Mark Twain, and The Way We Live Now by Anthony Trollope as alternatives.
Anything by Robert Anton Wilson - fiction or fact, who can tell?
For non-fiction, I would suggest:
Anatomy of Criticism by Northrop Frye - as important a refernce as Anatomy of Style by Strunk & White (which all readers & writers should memorize and keep close at hand)
Improvised News: A Sociological Study of Rumor - Tomatsu Shibutani
The Sacred and The Profane by Mircea Eliade
The Rise and Fall of the Roman Empire - Gibbons
The Rise and Fall of the Third Reich - William Shirer
Last, and by far not the least, perhaps the 2 books that most impacted my psyche (for good or bad) one "fiction", the other "non-fiction" but both blur in the same historical facts:
The Theory and Practice of Hell by Eugene Kogan
The Painted Bird by Jerzy Kosinski