Web Connector 23.2

# Run Custom JavaScript on Web Pages

The connector can run custom JavaScript on web pages before it crawls and ingests them. This allows you to manipulate pages or interact with them before they are processed. For example, if a page has a button to load more content you can configure the connector to run a script that clicks the button. The ingested page will then include the additional content.

TIP: Running custom JavaScript on pages is usually only necessary in advanced cases, such as processing unconventional sites.

You must write the custom script and save it to a file that can be accessed by the connector.

OpenText recommends that you enclose your script in a `try...catch` block to prevent exceptions being thrown out of the script. For example:

    try
    {
      // custom javascript
      true;
    }
    catch(e)
    {
      console.log("Javascript Error: " + e.message) false;
    }

Any messages that you write to the JavaScript console are written to the connector's synchronize log, but only when you set the parameter `LogLevel` to `FULL` and `EnableJavascriptConsoleLogging` to `TRUE`. To troubleshoot your script, set these parameters and include a reasonable amount of logging in your script. The connector also logs the return value of the script.

To run custom JavaScript on web pages

1. Stop the connector and open the configuration file.

2. Modify your fetch task by adding the parameter `CustomJSSections`. This parameter specifies the names of the sections in the connector's configuration file that contain settings for running custom JavaScript on pages. For example:

       [MyTask0]
       ...
       CustomJSSections=MyCustomJS

3. Create a new section in the configuration file to contain the settings for running the script:

       [MyCustomJS]

4. In the new section, set the following parameters:

   `CustomJSUrlRegex` | A Perl-compatible regular expression to match the full URLs of the pages on which you want to run custom JavaScript.
   `CustomJSScriptPath` | The path of the file that contains the custom JavaScript to run. The path must be absolute, or relative to the Web Connector executable file.
   `CustomJSElementSelector` | (Optional) The page elements on which to run the custom JavaScript. Specify the elements using CSS selectors.
   `CustomJSAllSelected` | (Optional) A Boolean value that specifies whether to run the custom JavaScript on all elements that match the selector specified by `CustomJSElementSelector` (the default value, `false`, means that the connector runs the JavaScript only on the first matching element).

   For example:

       [MyCustomJS]
       CustomJSUrlRegex=http://www\.hpe\.com/.*
       CustomJSScriptPath=javascript/my_script.js
       CustomJSElementSelector=div#maincontent

5. Save and close the configuration file.

Copyright 2023 Open Text
Version 23.2 | Last updated March 2023

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to swpdl.idoldocsfeedback@microfocus.com.

We appreciate your feedback!

Product: IDOL Web Connector 23.2

Topic Title: Run Custom JavaScript on Web Pages