Importing into CiviCRM using an import script
There are occasions where you want to get some data into CiviCRM but there is no quick way of doing it. For example, your existing contacts may all be tagged. You want to get these tag values into CiviCRM so that when you import your contacts your tags work properly.
The CiviCRM interface only allows you to add one tag at a time. So this could be very time-consuming if you have hundreds of tags. This recipe introduces the use of the command-line interface to rapidly add data to CiviCRM.
The recipe can be used to migrate data into most CiviCRM tables. It's not as terrifying as it sounds.
Getting ready
First, you must have a local testing environment.
A local testing environment is a CiviCRM installation that runs on your own computer rather than the Internet.
In this recipe our local testing environment was set up on a Mac using the MAMP software. There are similar setups for Windows-based machines.
Once your local testing environment is set up you need to be able to execute PHP from the command line. PHP is the scripting language that powers CiviCRM. We want to be able to type in a command that will run a PHP file that will control our data import.
Navigate to where MAMP stores the PHP executable file, normally /Applications/MAMP/bin/php/php5.3.14/bin/php. Make a note of this path.
Now let's open the Terminal application and enter the following command:
open -a TextEdit .bash_profile
Hit the Enter key.
This makes the TextEdit application open the .bash_profile file, which is normally a hidden file. Now let's add the following line to .bash_profile.
alias phpmamp="<path> "
Here, path is the path to the PHP executable file.
In this example the line reads:
alias phpmamp="/Applications/MAMP/bin/php/php5.3.14/bin/php"
Now save the file.
What this means is that when you type phpmamp into the Terminal application it will execute the version of PHP held in the MAMP installation.
Test it by going to the Terminal application and typing phpmamp –help.
If all is well, it will return a list of help options for PHP.
How to do it…
We will prepare a CSV file for import, and call the import.php CiviCRM file using a command-line interface.
- Prepare tag data as a
.csvfile. Each tag must have a name and a description. Your file might look like this:name,description Learner, People who join to learn things Influencer, People who want to influence our organization Participator, People who want to attend our events Status booster, People who gain professional status Like minders, People who agree with our politics Passives, People who just want our resources
- Open the Terminal application and navigate to the
site modulefolder that contains your CiviCRM installation in your local testing environment. This is easy. - Navigate to the CiviCRM module folder using the Finder and make sure it is visible when you use Terminal. Then type
cdinto the Terminal application window. Then drag thecivicrmfolder onto the Terminal window. This accomplishes the navigation with the minimum of work:
- Enter the command to import the data. In the Terminal application, enter:
phpmamp bin/csv/import.php -e Tag --file <path_to_file>
Here,
path_to_fileis the place where you stored your CSV file. You can grab this path by dragging the file from the Finder into the Terminal window.In this example the finished text will be:
phpmamp bin/csv/import.php -e Tag –file /Users/tonyhrx/Sites/book/tmp/tags.csv
- Copy and paste this into the Terminal window and press return. You should get the following result:
How it works…
phpmamp invokes PHP to run bin/csv/import.php.
-e tells the import script what CiviCRM entity we are going to target, in this case Tag.
--file is the absolute path to the file of data we are going to import, in this case /Users/tonyhrx/Sites/book/tmp/tags.csv.
Then the CiviCRM API does the rest.
There's more…
This simple but powerful technique can take hours off import times and accomplish most of your import routines.
You can explore the CiviCRM API on your own installation. Just type in civicrm/api/explorer after your domain URL:
If we look at the Tag entity we can see what fields it contains by selecting getfields from the actions drop-down menu.
You can do this for any entity that is listed.
For example, let's look at the Relationships entity and see what the fields are:
Using this, we could construct another CSV file and add relationship data to it such as:
name_a_b,label_a_b,name_b_a,label_b_a,description,contact_type_a,contact_type_b,is_reserved,is_active Secretary,Secretary,Secretary is,Secretary is,Used for Organization secretaries,Individual,Organization,0,1
And then run the import using Terminal:
phpmamp bin/csv/import.php -e RelationshipType --file /Users/tonyhrx/Desktop/relationships.csv
See more
- Refer to Chapter 11, Developing for CiviCRM for more information
- Find out more about this technique at http://civicrm.org/blogs/xavier/api_batch_tools