Modster, who entered the season as a candidate for the starting job but ended up as the No. 3 quarterback, posted on Twitter that he will continue his football career elsewhere.
"Excited for a new chapter!" Modster wrote.
The redshirt sophomore acknowledged former Bruins coach Jim Mora and current coach Chip Kelly.
"I'd like to thank Coach Kelly and the strength coaches for continuing to allow me to continue to play the game I love, BUT I've decided to leave UCLA and pursue other opportunities," Modster said.
Modster was beaten out for the starting job by Michigan graduate transfer Wilton Speight and freshman Dorian Thompson-Robinson in training camp. Speight was injured in the Bruins' season opener against Cincinnati and was replaced by Thompson-Robinson.
Thompson-Robinson started last week's game against Fresno State, and Modster played just one play in the 38-14 loss to the Bulldogs.
Speight might be available to return when UCLA (0-3) plays at Colorado (3-0) on Sept. 28 in its Pac-12 Conference opener.
Last season, Modster completed 64.6 percent of his passes for 671 yards and four touchdowns and no interceptions as the backup to Josh Rosen, a 2018 NFL first-round draft pick. The 6-foot-1, 215-pound Modster directed UCLA on a game-winning drive last year against California that locked up a bowl berth.
Then in a Cactus Bowl loss to Kansas State, Modster threw for 295 yards and two touchdowns.
With Modster departing, redshirt sophomore Matt Lynch, redshirt freshman Austin Burton and freshman walk-on Josiah Norwood remain as backups.
Speight wished Modster well wherever he lands.
"All love bro!" Speight tweeted. "Can't wait to watch you succeed."
Modster's Twitter post read:
"First and foremost I would like to thank my teammates for always grinding with me day in and day out, creating memories that I will cherish for a lifetime. I would like to thank Coach (Jim) Mora for giving me the opportunity to play and attend such a prestigious school! Also I'd like to thank Bruin Fans who continued to support my teammates an I throughout these years."
Modster must sit out next year as a transfer if he chooses an FBS school before becoming eligible to play again in 2020.