I think it's really up to you and how your body is doing. If you feel great, aren't sore and are motivated to perform the next day, go for it. Because everyone's body is a little different, you may need much less recovery time than other people here. Basically, just listen to your body, it'll let you know.
Just remember though, rest and recovery time are just as important for muscle development as lifting weights and exercising.