Before starting, make sure you have set up your GeoZones. Once that is done, you're ready to set up your tax rates!
First, navigate to System -> Localisation -> Tax Classes. Next you'll click the "Add" button. Then enter the "Tax Class Title" and "Description" for this tax rate. This is just for you, so name this tax class something you will easily recognize. Next click "Add Tax Rate" Select the GeoZone you'd like to apply this tax rate to. "Priority" come to play its role when the shipping address applies to two or more than two of your tax rates. For example, you have set up two tax rates, one for the United States, the other for New York City. If a customer comes from New York City, then priority of the tax rate will decide how much tax he has to pay. Then just fill out the form and click save. You can enter as many different tax rates as you'd like.
Once you have set up your tax rates, make sure to go back to your products and apply the tax rate to each one. Because products can all have different tax rates, this will need to be done individually. If you forget to apply the tax rate to the product, it will be sold tax free.