12.1. Importing data from a single CSV file

12.1.1. Using PERIODIC COMMIT

This tutorial will show you how to import data from a single file using LOAD CSV.

In this example, we’re given a CSV file where each line contains a mapping of an actor, a movie they’ve played in and the role they were cast as in that movie. Below we will assume that the CSV file has been stored on the database server and access it using a file:// URL. Alternatively, LOADCSV also supports accessing CSV files via HTTPS, HTTP, and FTP.

Using the following Cypher query, we’ll create a node for each movie, a node for each actor and a relationship between the two with a property denoting the character name. We’re using MERGE for creating the nodes so as to avoid creating duplicate nodes in the case where the same actor or movie appears in the file multiple times.

If the input CSV file is big enough, a good way of speeding up the MERGE commands is by creating two indices for Person nodes on the name property and Movies nodes on the title property before the LOAD CSV command.

Query. 

CREATE CONSTRAINT ON (p:Person) ASSERT p.name IS UNIQUE

Result

Constraints added: 1

(empty result)


Query. 

CREATE CONSTRAINT ON (m:Movie) ASSERT m.title IS UNIQUE

Result

Constraints added: 1

(empty result)


Query. 

LOAD CSV FROM
  "file:/jenkins/workspace/neo4j-2.1-release-candidate/target/community/cypher/cypher-docs/target/docs/dev/ql/import/csv-files/roles.csv"
  AS csvLine
MERGE (p:Person { name: csvLine[0]})
MERGE (m:Movie { title: csvLine[1]})
CREATE (p)-[:PLAYED { role: csvLine[2]}]->(m)

Result

Nodes created: 5
Relationships created: 6
Properties set: 11
Labels added: 5

(empty result)


LOAD CSV also supports loading CSV files that start with headers (column names). In this case these header names may be used to address the values in a row via the LOAD CSV WITH HEADERS clause.

12.1.1. Using PERIODIC COMMIT

When importing lots of data, the transaction state might build up so much that queries run very slowly or even crash the JVM. By using PERIODIC COMMIT, the transaction will be committed at periodic intervals.

All you need to do is start your query with the PERIODIC COMMIT hint, like so: .Query

USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM
  "file:/jenkins/workspace/neo4j-2.1-release-candidate/target/community/cypher/cypher-docs/target/docs/dev/ql/import/csv-files/movie_productions.csv"
  AS csvLine
MERGE (c:Country { name: csvLine.country })
MERGE (m:Movie { title: csvLine.movie })
CREATE (p)-[:PRODUCED { year: toInt(csvLine.year)}]->(m)

Result

Nodes created: 7
Relationships created: 3
Properties set: 7
Labels added: 4

(empty result)


LOAD CSV produces values that are collections (or maps when WITH HEADERS is used) of strings. In order to convert them to appropriate types, use the built-in toInt and toFloat functions.