RSelenium and Java Heap Space
I’m in the process of deploying a scraper on a DigitalOcean instance. The scraper uses RSelenium with the PhantomJS browser. I ran into a problem though. Although it worked flawlessly on my local machine, on the remote instance it broke with the following error:
Clearly Java a memory issue.
Since the Selenium server is being launched from within R, I did not have direct access to the
java command line options. However, setting an environment variable to increase the heap space resolved the problem.
The scraper is now chugging along happily and I’m moving on with my day.