How do I import CSV file into a MySQL table?
I have an unnormalized events-diary CSV from a client that I'm trying to load into a MySQL table so that I can refactor into a sane format. I created a table called 'CSVImport' that has one field for every column of the CSV file. The CSV contains 99 columns , so this was a hard enough task in itself:
CREATE TABLE 'CSVImport' (id INT); ALTER TABLE CSVImport ADD COLUMN Title VARCHAR(256); ALTER TABLE CSVImport ADD COLUMN Company VARCHAR(256); ALTER TABLE CSVImport ADD COLUMN NumTickets VARCHAR(256); ... ALTER TABLE CSVImport Date49 ADD COLUMN Date49 VARCHAR(256); ALTER TABLE CSVImport Date50 ADD COLUMN Date50 VARCHAR(256);
No constraints are on the table, and all the fields hold VARCHAR(256) values, except the columns which contain counts (represented by INT), yes/no (represented by BIT), prices (represented by DECIMAL), and text blurbs (represented by TEXT).
I tried to load data into the file:
LOAD DATA INFILE '/home/paul/clientdata.csv' INTO TABLE CSVImport; Query OK, 2023 rows affected, 65535 warnings (0.08 sec) Records: 2023 Deleted: 0 Skipped: 0 Warnings: 198256 SELECT * FROM CSVImport; | NULL | NULL | NULL | NULL | NULL | ...
The whole table is filled with
I think the problem is that the text blurbs contain more than one line, and MySQL is parsing the file as if each new line would correspond to one databazse row. I can load the file into OpenOffice without a problem.
The clientdata.csv file contains 2593 lines, and 570 records. The first line contains column names. I think it is comma delimited, and text is apparently delimited with doublequote.
When in doubt, read the manual: http://dev.mysql.com/doc/refman/5.0/en/load-data.html
I added some information to the
LOAD DATA statement that OpenOffice was smart enough to infer, and now it loads the correct number of records:
LOAD DATA INFILE "/home/paul/clientdata.csv" INTO TABLE CSVImport COLUMNS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES;
But still there are lots of completely
NULL records, and none of the data that got loaded seems to be in the right place.
The core of your problem seems to be matching the columns in the CSV file to those in the table.
Many graphical mySQL clients have very nice import dialogs for this kind of thing.
My favourite for the job is Windows based HeidiSQL. It gives you a graphical interface to build the
LOAD DATA command; you can re-use it programmatically later.
To open the Import textfile" dialog, go to
Tools > Import CSV file:
Use mysqlimport to load a table into the database:
mysqlimport --ignore-lines=1 \ --fields-terminated-by=, \ --local -u root \ -p Database \ TableName.csv
To make the delimiter a tab, use
Read more... Read less...
Simplest way which I have imported 200+ rows is below command in phpmyadmin sql window
I have a simple table of country with two columns CountryId,CountryName
here is .csv data
here is command:
LOAD DATA INFILE 'c:/country.csv' INTO TABLE country FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 ROWS
Keep one thing in mind, never appear , in second column, otherwise your import will stop
I know that the question is old, But I would like to share this
I Used this method to import more than 100K records (~5MB) in 0.046sec
Here's how you do it:
LOAD DATA LOCAL INFILE 'c:/temp/some-file.csv' INTO TABLE your_awesome_table FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' (field_1,field_2 , field_3);
It is very important to include the last line , if you have more than one field i.e normally it skips the last field (MySQL 5.6.17)
LINES TERMINATED BY '\n' (field_1,field_2 , field_3);
Then, assuming you have the first row as the title for your fields, you might want to include this line also
IGNORE 1 ROWS
This is what it looks like if your file has a header row.
LOAD DATA LOCAL INFILE 'c:/temp/some-file.csv' INTO TABLE your_awesome_table FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 ROWS (field_1,field_2 , field_3);
phpMyAdmin can handle CSV import. Here are the steps:
Prepare the CSV file to have the fields in the same order as the MySQL table fields.
Remove the header row from the CSV (if any), so that only the data is in the file.
Go to the phpMyAdmin interface.
Select the table in the left menu.
Click the import button at the top.
Browse to the CSV file.
Select the option "CSV using LOAD DATA".
Enter "," in the "fields terminated by".
Enter the column names in the same order as they are in the database table.
Click the go button and you are done.
This is a note that I prepared for my future use, and sharing here if someone else can benefit.
Try this, it worked for me
LOAD DATA LOCAL INFILE 'filename.csv' INTO TABLE table_name FIELDS TERMINATED BY ',' ENCLOSED BY '"' IGNORE 1 ROWS;
IGNORE 1 ROWS here ignores the first row which contains the fieldnames. Note that for the filename you must type the absolute path of the file.