A common problem when working with files in SuiteScript was the 10mb file size limit. This either meant that functionality you wanted to implement was not possible or required a workaround. 

With NetSuite 2017.1 a new file streaming API has been added that removes the 10mb limit for creating and reading large CSV and plain text files. (Note: this functionality is only available in SuiteScript 2.0)

A few new functions have been added to the file module to help with this:

File.lines.iterator() //returns an iterator object that enables you to process each line in the file
File.appendLine(options) //adds a line to the end of the file
File.resetStream() //resets the file streams opened by iterator or appendLine functions

Creating a file with File.appendLine

In the example below we are creating a custom module that will create a CSV file in the NetSuite file cabinet based on the fileData object that is passed in. As long as each line in the file is less than 10mb there will be no issues creating a file that is greater than 10mb.

//Require the file module
define(['N/file'], function (file) {

  functon saveCSVFile(fileData) {
    // Create the CSV file
    var csvFile = file.create({
        name: 'data.csv', 
        contents: 'date,amount\n', 
        folder: 39, 
        fileType: 'CSV'
    });

    //use appendLine to append each line to the file
    for (var i = 0; i < fileData.length; i++) {
      csvFile.appendLine({
          value: fileData[i]
      });
    }

    //save the newly created CSV file
    var csvFileId = csvFile.save();
    return csvFileId;
  }

  return {
    saveCSVFile:saveCSVFile
  };

});

Reading a file with an iterator

In this next example we are creating a module that will read in a CSV file, process it line by line and return a calculated total.

//Require the file module
define(['N/file'], function(file) {

    functon readCSVFile(csvFileId) {
        var csvFile = file.load({
            id: csvFileId
        });
        var iterator = csvFile.lines.iterator();

        //Skip the first line (CSV header)
        iterator.each(function() {
            return false; //return false to only process the first line
        });

        var total = 0;

        iterator.each(function(line) {
            //split each line by the column delimiter
            var lineValues = line.value.split(',');

            //do something with the data
            var lineAmount = parseFloat(lineValues[1]);
            total += lineAmount;

            //return true to continue to the next line
            return true;
        });

        return total;
    }

    return {
        readCSVFile: readCSVFile
    };

});

Summary

The simple examples above just cover the basics of the new API, but combining these with the map reduce script type enables you to have some very powerful processing of large files. If the CSV import functionality in NetSuite is too restrictive when importing your data, these API's offer a more powerful and customisable solution.

 

To stay in the loop for further news, guides, resources and more, sign up to "insight" our flagship weekly NetSuite and Business Software Newsletter, click here.

 

4 Comments