You do not need to uninstall the extensions before updating them. In fact, by uninstalling them, you will lose all configurations (module instances...).
Before updating, make sure you read the change logs and are aware of potential backward compatibility issues.
Moving to Joomla 4 from Joomla 3.10
For a successful migration to Joomla 4, the extension first needs to be up to date in your Joomla 3.10 configuration.
For general instructions on moving from Joomla 3.10 to Joomla 4, please check this tutorial out!
Must read: specific instructions about the migration for this extension.