Start a new topic

Importing products from external database

Following client request, we will need to implement a service that will import products data from client's database (sql server) into Virto database.

I have the following assumptions:

1. Doesn't have to be real time or synchronised, could be done once a day

2. Products database contains thousands of products, several prices lists (different currencies), several languages


As far as I understand, there could three general methods solving this challenge:

1. Creating csv files from clients database and then importing these csv files: catalogs, properties, prices, etc. (could not find a way to automate this service, though)

2. Using WebAPI - each night delete the Virto products database, and re-populate it.

3. Updating directly Virto database using sql scripts or stored procedures.


Does anyone got any good advice / comment / thoughts ?

Cheers




Hi Ran,


all approaches are valid. 


Here are some questions:

  1. Do you plan to host solution in azure or other services?
  2. Is all product data will be managed in external database, or do you expect some data like relations, images to be only in VC while description, pricing be in that external system?
  3. Is sync one or two way?


Now more to the approaches:


  1. This is an ok approach, but it will require extra effort to generate CSV's and then creating an import services that processes those CSV's in reliable way (I wouldn't really recommend this).
  2. That is a good approach, but will require some coding. You also probably want to update existing products instead of clearing up the database all together. This can be implemented as a hangfire service that runs in the background.
  3. If you want to update database directly, then the fastest and preferred approach will be to use SSIS (SQL Server Integration Services), you can create transformation packages that transform the data from your sql database to VC database. It also includes all the tools for monitoring and logging. This is the fastest way to accomplish the task.

Hope this helps,
Sasha.

1 person likes this

@Sasha, thanks for the quick response. It sure helped.


1. The solution will be probably stored on our servers

2. All data/product data will be taken from external database (images will be probably stored on a third server or cloud based solution) 

3. For now it will be only one way (client products database => Virto database)



By images, I meant does this external system contains information about the images and how they are associated with products. Also does it contain relations, that can be used for cross sell, upsell, related products and so on. Basically is it a full PIM?


In anyway, SSIS would be the fastest way to accomplish this import.

Login or Signup to post a comment