In an official statement, Tidal said, "we are thankful to Peter for stepping in as interim CEO and wish him the best for the future.
"Tidal, will be transitioning to a permanent CEO as part of our strategic plan to create a leading platform, and current executives in New York and Oslo will continue to lead our rapidly developing innovation and content initiatives until our new CEO is in place."
It's been a turbulent few months for Jay-Z's music-streaming service. Tonstad is the second CEO to part with the company this year - he follows in the footsteps of previous boss Andy Chen, who left Tidal back in April.
Tidal has come in for some criticism since its star-studded US relaunch which was judged to have put too much focus on the superstar-backers who include include Madonna, Beyonce and Kanye West.
The news on Tonstad's departure comes just days before Apple's rival music-streaming service, Apple Music, rolls out to consumers.
MORE: Can Tidal take high-quality audio to the mainstream?
MORE: Apple Music streaming service - everything you need to know