Command Line Import Multiple Files

Discussion of open issues, suggestions and bugs regarding database management and administration tools for MySQL
Post Reply
phcaptjim
Posts: 4
Joined: Wed 12 Sep 2012 12:08
Location: Philadelphia, PA
Contact:

Command Line Import Multiple Files

Post by phcaptjim » Wed 12 Sep 2012 12:34

I am trying to load multiple XML files that have the same schema and same mapping. These files are scattered throughout sub folders for months and days. The following COMMAND shows me the list of files that I am looking for (the /s is to check all sub folders):

Code: Select all

"DIR C:\FINANCE\XML\2012\bank.xml /s"
I have a file called bank.xml inside of subfolders like month/day/etc and I want to pull all of those files into the MYSQL db. How would I do that using the /inputfile parameter?

I have tried this but it returns an error message:

Code: Select all

/inputfile:"C:\FINANCE\XML\2012\bank.xml /s"
Any ideas?

Thanks so much!
Last edited by phcaptjim on Wed 12 Sep 2012 15:05, edited 1 time in total.

alexa

Re: Command Line Import Multiple Files

Post by alexa » Wed 12 Sep 2012 13:49

Please follow the procedure:

1. Start the GUI of dbForge Studio for MySQL.
2. Right-click the table you want to import data to and select 'Import Data' from the popup menu. The Data Import wizard opens.
3. Complete all the pages of the wizard including the last one 'Errors handling' page.
4. Click the 'Save template' button in this wizard and save the template to a file (for example, "sample_template.dit").
5. Start command line interface of dbForge Studio for MySQL and type the following:

dbforgemysql /dataimport /templatefile:"sample_template.dit"

phcaptjim
Posts: 4
Joined: Wed 12 Sep 2012 12:08
Location: Philadelphia, PA
Contact:

Re: Command Line Import Multiple Files

Post by phcaptjim » Wed 12 Sep 2012 14:46

That process works great for loading one XML file. I'm trying to script this to load multiple XML files as it would be very time consuming to load each file individually when we are talking about 1000's of files. Is there a way to accomplish this? It seems that getting the right syntax should allow multiple files to be loaded.

alexa

Re: Command Line Import Multiple Files

Post by alexa » Wed 12 Sep 2012 15:51

Your task with multiple files can be automated by creating the *.bat file below:

-----
set apppath="D:\Program Files\Devart\dbForge Studio for MySQL\dbforgemysql.com"

rem export of file paths to tables.txt
%apppath% /dataexport /templatefile:exptmpl.det
rem If export succeeds then start import in cycle
if ERRORLEVEL 0 @(
for /F %%i in (tables.txt) do @(
%apppath% /dataimport /templatefile:imptmpl.dit /inputfile:"%%i"
)
)
-----

Please note that exptmpl.det and imptmpl.dit files have to be preliminary prepared with the Data Export and Data Import wizards respectively (please refer to our post). Please also make sure you specify 'tables.txt' in the 'File name' field on the 'Export format' page of the Data Export wizard when preparing exptmpl.det

phcaptjim
Posts: 4
Joined: Wed 12 Sep 2012 12:08
Location: Philadelphia, PA
Contact:

Re: Command Line Import Multiple Files

Post by phcaptjim » Wed 12 Sep 2012 19:15

Thank you for the excellent support on this problem. Your solution works great and is exactly what I was looking for. I'm proud to be a new customer!

alexa

Re: Command Line Import Multiple Files

Post by alexa » Thu 13 Sep 2012 09:28

Please let us know should other questions arise.

phcaptjim
Posts: 4
Joined: Wed 12 Sep 2012 12:08
Location: Philadelphia, PA
Contact:

Re: Command Line Import Multiple Files

Post by phcaptjim » Fri 14 Sep 2012 16:57

I have one more question concerning this same project.

Some of the XML files that I am loading do not have a primary key that I can use to link them back to other XML files that are getting loaded into the database. What I would like to do is take the file path / name that is stored in the %%i variable in the example above and attached that to each record when loading the XML data. Is something like this possible? Can a variable such as that be mapped to a field in the database?

Thanks!

alexa

Re: Command Line Import Multiple Files

Post by alexa » Mon 17 Sep 2012 11:42

Unfortunately, we have no solution for such a task at the moment.

However, you could add your suggestion on our UserVoice forum so other users can vote for it: http://devart.uservoice.com/forums/7729 ... -for-mysql

Post Reply