File Upload getting

2 followers
0
Avatar

I was in the process of uploading a csv file to Datameer. My first file worked but the second one gets stuck at 20%. I tried refreshing and restarting the application but it is still getting stuck. Any thoughts on how to resolve?

Thanks!

Stanley

Stanley

4 comments

  • Avatar
    Joel Stewart

    Hi Stanley, what type of file is it and what size is the file? 

    If you're able to look in the <INSTALLDIR>/logs/conductor.log file, perhaps there are some errors or warnings that could explain the slowness you're observing. 

    0
  • Avatar
    Stanley

    Hi Joel, So looked through the documentation and realized that you can only upload upto a 500MB file. My file was larger than that. 

    Is my next best option to try an Import Job? The data is in a csv file that I can load to a MSSQL database.

    0
  • Avatar
    Joel Stewart

    Yes, an Import Job is a must more effective way to handle data sets larger than 500 MB including data sets into the TB and PB size. 

    CSV is an okay format for an Import Job (as is MySQL). You'll just want to create a Connection in Datameer first. If you can connect via SSH or SFTP then leaving the file as a raw CSV seems best. If a JDBC connection to MySQL is the easiest option in your environment, this is suitable as well. 

    0
Please sign in to leave a comment.