Doctors recommend washing your hands around 10 times a day.
It is important to wash your hands with running water and soap and minimize the times when you're using hand sanitizer, as the latter cannot clean the hands well enough and protect you from infectious disease, and it has a few other drawbacks, too. However, if you don't have access to running water, hand sanitizer is a good temporary way to clean your hands, at least partially.
Another point to keep in mind is that antibacterial soap doesn't offer any additional benefits compared to regular soap.