Map/Reduce Scheduler to Prevent Concurrency Race Conditions

For most map/reduce scripts created in NetSuite, running multiple instances of the script concurrent with each other should not be a problem. However, if the map/reduce instances have a chance of reading the same records and editing them at the same time, you may run into what is called “race conditions,” an issue with concurrency where the result may be erroneous and inconsistent based on how the instances run simultaneously with one another.

About this Solution to the Map/Reduce Script Concurrency Issue

This setup will prevent race conditions from occurring with your map/reduce script by forcing each instance to run one at a time without dropping any instances that attempt to run while the map/reduce is in progress.

For the steps below that involve code, I will paste code blocks from one of my examples and add annotations explaining each step.

Setting up a Map Reduce Script to Prevent Race Conditions

Step 1: Create a custom record that contains all of the parameters that your map/reduce script would need. We will refer to this custom record as a “queue record.”

Step 2: Create and deploy a scheduled helper script that calls the map/reduce script using the newest of your queue record instances. Here is an example:

/** * @name AG_SS_UpdateETA_Helper.js * * @author Anchor Group : John Baylon * @version 1.0.0 * @since 2023-12-7 * * @file read from update ETA queue and execute a map/reduce script * * @NApiVersion 2.1 * @NScriptType ScheduledScript */ define(["N/search", "N/task"], function (search, task) { function execute(context) { const queueSearch = search.create({ type: "customrecord_updateeta_queue", filters: [], columns: [ search.createColumn({ name: "created", sort: search.Sort.ASC, label: "Date Created" }), search.createColumn({name: "id", label: "ID"}), ] }); const result = queueSearch.run().getRange(0, 1); if (result.length > 0) { // attempt to call map reduce script try { const mrTask = task.create({ taskType: task.TaskType.MAP_REDUCE, scriptId: "customscript_ag_mr_updateeta", params: { custscript_updateeta_id: result[0].getValue("id") } }); mrTask.submit(); } catch (e) { // error expected to be thrown if map/reduce script is already running // print error in case if error is something else unexpected if (e.name !== "NO_DEPLOYMENTS_AVAILABLE") { log.debug("error thrown starting map reduce", e); } } } } return { execute: execute }; });

The queue search is to find the oldest queue record that has yet to be processed. If no queue records are found, the scheduled script ceases to run. Otherwise, the scheduled script will attempt to run the map/reduce script using the oldest queue record. There is a section for handling a “NO DEPLOYMENTS AVAILABLE” error. This will be explained in the next step.

Step 3: Reduce the number of deployments for your map/reduce script to a single deployment. Additionally, decrease the concurrency limit in the script’s deployment to 1. This will force each thread of the map/reduce script to run one at a time. If the scheduled script attempts to start the map/reduce script while it is already running, it will throw an error while continuing to process the data it is already processing.

Step 4: Modify your map/reduce script to read from the queue record instead of the parameter data. The map/reduce script’s parameters can be reduced to a singular text field that contains the queue record’s ID at a given time. The script should read from the queue record, as it contains the parameter data needed for the map/reduce script to run.

Step 5: Add the following to the summarize step of the map/reduce script:

The summarize step only runs when the map/reduce script has completed running. The queue record it worked on will be deleted, as it is no longer needed. Then, it will call the scheduled helper script to run, which may trigger the map/reduce script with the next set of queue record data to process off of.

Step 6: Modify your user event script that triggers the map/reduce script to instead create a queue record and populate it with what would have been the map/reduce parameters, and then start the scheduled helper script. Below is an example of code that would be added to the end of a user event script:

// create the queue record const queueRecord = record.create({ type: "customrecord_updateeta_queue", }); queueRecord.setValue({ fieldId: "custrecord_updateeta_items", value: sendString }); queueRecord.save(); // run the scheduled helper script const scheduledScript = task.create({taskType: task.TaskType.SCHEDULED_SCRIPT}); scheduledScript.scriptId = 4693; scheduledScript.deploymentId = "customdeploy_ag_ss_updateeta_helper"; scheduledScript.submit();

Creating a queue record is like adding a ticket for the map/reduce script to process once it has completed all other tickets ahead of the one just created. Running the scheduled script either will wake up the map/reduce script if it is not running at the time, or it will attempt to start another instance of the map/reduce script and fail, which is an intended result.

Got stuck on a step in this article?

We like to update our blogs and articles to make sure they help resolve any troubleshooting difficulties you are having. Sometimes, there is a related feature to enable or a field to fill out that we miss during the instructions. If this article didn't resolve the issue, please use the chat and let us know so that we can update this article!

Oracle NetSuite Alliance Partner & Commerce Partner

If you have general questions about SuiteCommerce or more specific questions about how our team can support your business as you implement NetSuite or SuiteCommerce, feel free to contact us anytime. Anchor Group is a certified Oracle NetSuite Alliance Partner and Commerce Partner equipped to handle all kinds of NetSuite and SuiteCommerce projects, large or small!

Tagged with Training