Should you lose your slippery little, impossible to handle unless you have baby hands, Chromecast with Google TV remote, Google has replacements for you. Listed this morning, you can buy the new Voice Remote for Chromecast With Google TV (official name) for $19.99 in all three colors.
The remote still only works with the new Chromecast, but having them available as a separate purchase is a smart move by Google. Losing one aside, you could also consider picking one up if you wanted a different color than the one that came with your dongle. Maybe you bought white and want a blue remote, or maybe you went blue and need a little sister remote to go with it, now you can.