I'm not going to immediately take the baby's temp
Why wouldn't you do that?
I want to let him sleep a bit more before I start with the test
I think it's important for you to want to monitor the health of your child.  However, I totally understand you wanting to let them rest more first.  It's good not to rush things unless you have to.
I'm not such an anxious person, but this anxiety is just natural when you have kids.
It is understandable that your child being sick can make you even more anxious. Children can get sick so easily that it seems like we always have to look after their health.