Main content begins here

Import & Delete Half A Million Drupal Nodes

Thursday, July 21, 2011

A recent project required functionality to import and delete half a million Drupal nodes. In our case the client had vendor-supplied CSV files of store location information. The CSV files were not Drupal-ready so we developed our own custom script to ready the original CSV for import. Each store chain had its own CSV so we could control the size of the files during import.

The Feeds module was used to import the CSV as Drupal nodes, complete with store address and product information. The store address was required to Google map the locations in a Store Finder on the site. The Location and GMap modules provided the mapping we needed. Views and the Views Attachment provided the dual-results we needed, where one result set was the Google map and another result set was rendered as a list below the map.

Another hurdle was the deletion of nodes. We initially tried the CSTools solution of using "content creation" in reverse, meaning we set it to "delete all content" before creating new content, then set the content creation amount to zero While this worked at first, once we were in the half-million node area it ceased to work with an obscure error.

We then tried the Bulk Delete module which we had overlooked previously because it says it is a slow module. The module did also fail but provided us with a more helpful error, leading us to increase our MySQL packet size. Bulk Delete did work then, taking a whopping 5 hours to delete @ 450,000 nodes

Post new comment

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <p> <h2> <br> <img>
  • Lines and paragraphs break automatically.

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.

You ROCK!!

J. G., Owner GoForChange.com
Copyright ©2001-2017 Way Cool Web Design LLC. All Rights Reserved.