Baby Birds, did I screw up?

I have a pair of Zebra Finches that mated. The eggs hatched about 4 weeks ago. I opened the nest box to check on them, and all four flew out. I had to chase them down and catch them to put them back. Since my scent got all over them, did the mother decide to reject them? They are about 4 weeks old and have all their feathers, but their beaks are still black and they don't really come out of the nest box. Did the mother abandon them because i touched them?