You have to just upload the plugin and in network settings, check the option to enable plugin management, and then from each individual site you need to activate it.