The band was formed in 2004 as a solo project of singer Ryan Dunson. Dunson signed with Vice Verse Virtue Records, a Tennessee record label, in late 2002. Soon after, Dunson signed to One Eleven Records, releasing a 1000-copy EP called Late Night Makeout. In 2004 One Eleven Records released the band's first record, Having to Let Go, as a 13-track LP. In 2005, OneEleven Records re-released the album as a ten-track LP. After expanding to a five-piece, their second album, The Goodnight Moon, was released in April 2006.[1][2] Following this release, they toured through North America on the Warped Tour,[3] as well as with groups like Relient K, Mae, Secondhand Serenade, PlayRadioPlay, The Graduate, My American Heart, The Rocket Summer, Dave Melillo, Cute Is What We Aim For, Love Me Electric, Blackpool Lights, Four Letter Lie, As Cities Burn, We Are the Fury, Scenes from a Movie, MUTEMATH, Copeland, Mest, Quietdrive, Under the Influence of Giants, Upper Class Trash and label mates The Spill Canvas, This Day & Age, and Rory.