Extremely large data bulk via php

Hello,

after a long time without posting anything here, I come with a post that might result usefull to some people.

The thing is that I found myself with a xls sheet with about 50,000 registers, that my client needed uploaded to his personal server, he gave extra-restricted access to a ftp server and phpMyAdmin, and he just wont understand how hard is to upload that amount of registers without seeing a "fatal error" about memory limits or timeouts.

Maiking it quick and to the point, if you have a large execution php script, and by large I mean by size and time, this two codelines can save your life:

ini_set('memory_limit', '-1');
set_time_limit(0);

yes, you gussed it, the first line is to override and set no memory limit, and the second one is to override the time limit.

you need to be responsable using this, because depending on the power of your server your site can slow down significantly while processing the bulk load.

the other option is to get navicat pro this little badass will transform your xls sheets to a MySQL valid file, oh yeah!, then you can use phpMyAdmin, to upload it, for this I recomend to zip the generated script, and use the import option, of course you will get a timeout, memory exausted and shit, BUT the cool thing about phpMyAdmin is that it will automatically continue where it left if you upload the file again without navigating to other page.

Well, I hope you can migrate your data succesfully :)

Happy Coding!