export.pl

Quick Preface: Many of my friends and community members have written their own custom processors, or tools to give them powerful export capabilities from ServiceNow. Crossfuze’s Scheduled Data Extract, Jason Petty’s FTP Export tool, and others come to mind. However, the purpose of this article is to focus on using the Out-of-Box export interface that uses no other technology other than the HTTP Requestor (a Perl script, in this case).

The Basics of Chunking an Export

ServiceNow offers a number of out-of-box ways to export its data. However, other than the ODBC driver, all of the export mechanisms have record limits placed on them so as not to bring down the instance when exporting a large number of records from a table.

The XML Web Service, however, does have some capabilities that allow you to easily and reliably chunk the requests using paged queries. Using the wiki documentation on exporting large amount of data, you can get the general feel for how to leverage the XML web service processor to export the full contents of a table in a paged manner so as to use several smaller requests that don’t bring the instance to its knees.

Essentially, through a series of request parameters, you sort the data on a field in the table. In most cases this would be the sys_id field. You also give it a max number of records to return in a given request. Finally, you specify a beginning value to start with in each given request (eg. use the last value of the sorted field from the previous request made). Optionally, you can choose to have the data exported in “unload” format that shows both Display Values as well as sys_id values. You can also filter the data before exporting it so that you only bring in data that matches certain criteria.

Any REST based client should be able to export data in this manner as long as it can read the response and generate a new request based on the last record that was brought over.

In order to show you how this is done, I have created a Perl Script/Utility that allows you to quickly and easily export data from a ServiceNow table into a file on your file system.

Perl-based Utility

The perl utility does require that you have Perl installed with some of the core libraries. Additionally, you will need the following Perl libraries that are available via CPAN:

  • LWP::UserAgent
  • HTTP::Request
  • XML::XPath

The script will accept the following command line arguements:

  • Table – The name of the ServiceNow table. (Not the table label, but the actual name)
  • Base Instance URL – This is the full URL to access the ServiceNow instance (eg. https://MyInstance.service-now.com)
  • Username – A username for a user that has privleges to access the table and its data
  • Password – The password for the user
  • Page Size – How many records should be returned with each web request. There is no hard and fast rule for this, however, I like to keep it around 100 or 1000. Test to see what works best for you.
  • Show Display Values – YES: show the display values for records in addition to the sys_id; NO – return only sys_id’s for reference field data
  • Encoded Query (optional) – An encoded query that filters the data in case you only want to export qualified data.

Running the script as simply: 

export.pl

will give you all of the parameters as well as some examples.

Here is the code. I tried to make a lot of comments so as to teach you what is going on under the covers: