I'm moving from one state to another, can I move my LLC with me?

Most taxes are based upon residence, where work is done, where goods are sold or where property is owned--i.e. taxes don't depend on the state where a business is established or registered as much as on more practical aspects of location. As one of my colleagues explains, there are advantages to being registered in the state you live/work in; typically, registering in a state involves paying some small fee(s), but there truly are small compared to other costs and taxes (a few tens or a few hundred dollars) and should not be the determining factor in your decision.

You do not necessarily need to to transfer the LLC.  If you are running your business the same way and not doing business in FL as stated in the statute relating to conducting business, you do not need to transfer the business.  You may however, fill out a form and file it with the secretary of state.  In some states this form is a certificate of authority.  The form permits you to sue companies in FL that may conduct business.  However, and like I said, if your LLC is still operating the same way and not doing business in FL, then you do not need to change anything.  I suggest speaking to a lawyer about the nature of your business and whether you are going to change it in any way that will require you to need to register in FL.  You should also speak to your accountant as you will need to pay income tax in FL and possibly a business entity tax in NY.