Tsitsipas struggled for only the second time in his career, falling behind by two places.
The number 2 from Spain was stunned when Tsitsipas won 3-6 2-6 7-6 (7-4) 6-4 7-5 in Melbourne.
Tsitsipas, 22, showed a wide smile after nearly converting his third match ball in disbelief.
Tsitsipas will face Daniil Medvedev in the semi-finals on Friday.
Medvedev, the fourth seed, defeated his countryman Andrey Rublev for the first time at the Australian Open
The 25-year-old won 7-5, 6-3, 6-2 while his childhood friend Rublev, the seventh-seeded player, struggled in a temperature of 29 degrees Celsius at Melbourne Park.
ATP finals champion Dmitry Medvedev continued his 19-match unbeaten run against Tsitsipas, who needed more than four hours to beat Nadal.
I am speechless, I have no words to describe what just happened in court, said Tsitsipas, who was still in an uproar when he spoke in court.
My tennis speaks for itself. It’s an incredible feeling to fight at this level and give everything on the field.
I got very nervous, I’m not going to lie. I don’t know what happened after the third set. I flew like a little bird.
It all worked out for me, and the feelings are indescribable.
Serenity Tsitsipas becomes a major for Nadal
Few could have predicted the outcome of the match between Tsitsipas and Nadal – let alone after the first two sets.
Tsitsipas seemed stunned by Nadal’s presence, clean strokes and direct play. Twenty unforced errors by the Greek allowed the Spaniard to take the lead.
The two met two years ago in Melbourne, when Nadal allowed Tsitsipas to play just six games in a one-way semi-final.
After that match, a depressed Tsitsipas declared that Nadal had a unique talent for making his opponent play badly.
At first it was déjà vu. This time, however, Tsitsipas remained calm and dragged Nadal into a physical fight that went over his head.
Tsitsipas made a lot of mistakes in the third set and although he was not worried about Nadal’s serving game, he found himself in a tiebreak when the Spaniard missed two shots on high and another backhand at crucial moments.
I’ve been very careful to stay calm and keep my nerves in check, that’s very important to me, Tsitsipas said.
I failed in some of my games. A big part of my victory was keeping my composure in the critical moments.
With his confidence growing, Tsitsipas continued to push Nadal back and put pressure on Nadal’s service game as the Spaniard began to miss serves.
Tsitsipas served out the fourth set after a critical break at 5-4 due to a forehand error by Nadal and then grabbed serve for 6-5 in tight conditions.
Nervousness struck as Nadal threatened to make the tiebreak a close game, but Tsitsipas managed to pull back the focus and seal a famous victory.
My moods were constant and I tried to keep things to myself. I’m happy with my attitude on the field, he said.
No excuses from Nadal as Federer exceeds rating
After dismantling Tsitsipas with powerful and accurate strokes in the first two sets, Nadal’s level dropped as his opponent advanced.
Nadal showed no signs of the back injury that ruled him out of the ATP Cup warm-up, but it remains to be seen whether lack of match practice – and perhaps his aging body – played a role in his young opponent getting ahead of him.
I can’t think of many excuses or reasons, said Nadal, whose only triumph in Melbourne was in 2009.
It was nothing special. I missed a few balls in the tiebreak that I shouldn’t have missed if I wanted to win. That’s all I’m saying.
I should be ready to play for four hours. I think my physical condition is doing well.
But it’s true that I probably needed some more time, because I was in top form and I couldn’t train for 20 days like I had planned, I couldn’t play any ATP Cup matches.
Only once before had Nadal lost a Grand Slam match after winning in the first two sets. After Tsitsipas’ amorous defeat, he looked dumbfounded as he sat head-shaking in his chair.
Although he regrouped and had a chance to escape and secure a tiebreak in the final, the Spaniard hardly complained.
Stefanos probably played better than me in the important moments. He added that it was a balanced match.
One of the best games I’ve played lately.
Daniil Medvedev is three times in the semi-finals of Grand Slam.
With the victory over Rublev, one of the strongest players on the ATP Tour last year, Medvedev extended his win streak against top 10 players to 11 matches.
The Russian duo came close in a hard-fought first set, alternating a break before Medvedev took the first set when the 23-year-old Ruble was broken on love.
Rublev was particularly affected by the humidity and was on the towel box after a grueling rally of 43 strokes.
Both players took a deep breath during the match, but it was Medvedev, who improved his fitness by getting into the top ten who was able to handle the conditions better.
Medvedev converted his first match point and reached the semi-finals of his third career Grand Slam, although he had a small problem when he called for a coach late in the match after cramping in his left hip.
It’s one of the best games I’ve played lately, Medvedev said. This game is really great.