Using a Validate Address Loqate Spark Job
-
Create an instance of
LoqateAddressingFactory
, using its static methodgetInstance()
. -
Provide the input and output details for the Validate Address Loqate job by
creating an instance of
LoqateAddressingDetail
specifying theProcessType
. The instance must use the type SparkProcessType. For this, the steps are:-
Configure the JVM initialization settings by creating an instance of
LoqateAddressingGeneralConfiguration
.Use the enum Enum ExhaustedAction. -
Configure the necessary database settings by creating an instance of
LoqateAddressingEngineConfiguration
and set the various fields. -
Configure the address validation settings by creating an instance of
LoqateAddressingValidateConfiguration
.To set the values of the various fields of this instance, use the enums Enum AcceptanceLevel, Enum CountryCodes, Enum OutputCasing, Enum CountryFormat, and Enum ScriptAlphabet. -
Set the details of the Reference Data path by creating an instance of
ReferenceDataPath
. See Enum ReferenceDataPathLocation. -
Create an instance of
LoqateAddressingDetail
, by passing an instance of typeJobConfig
, theLocalReferenceDataPath
instance, and theLoqateAddressingValidateConfiguration
instance created earlier as the arguments to its constructor.TheJobConfig
parameter must be an instance of type SparkJobConfig.- Set the details of the input file using the
inputPath
field of theLoqateAddressingDetail
instance.Note:- For a text input file, create an instance of
FilePath
with the relevant details of the input file by invoking the appropriate constructor. - For an ORC input file, create an instance of
OrcFilePath
with the path of the ORC input file as the argument. - For a parquet input file, create an instance of ParquetFilePath with the path of the parquet input file as the argument.
- For a text input file, create an instance of
- Set the details of the output file using the
outputPath
field of theLoqateAddressingDetail
instance.Note:- For a text output file, create an instance of
FilePath
with the relevant details of the output file by invoking the appropriate constructor. - For an ORC output file, create an instance of
OrcFilePath
with the path of the ORC output file as the argument. -
For a parquet output file, create an instance of ParquetFilePath with the path of the parquet output file as the argument.
- For a text output file, create an instance of
- Set the name of the job using the
jobName
field of theLoqateAddressingDetail
instance.
- Set the details of the input file using the
-
Configure the JVM initialization settings by creating an instance of
-
To create and run the Spark job, use the previously created instance of
LoqateAddressingFactory
to invoke its methodrunSparkJob()
. In this, pass the above instance ofLoqateAddressingDetail
as an argument.TherunSparkJob()
method runs the job and returns aMap
of the reporting counters of the job. - Display the counters to view the reporting statistics for the job.