Scheduled Jobs are not triggered by Datameer Scheduler
Scheduled Datameer artifacts are not being triggered by the scheduler.
One possible reason of this issue is a non-correct cron pattern for a scheduled execution that is saved in the Datameer database.
Job trigger settings for every artifact are stored in the Datameer application database in the table
pull_typekeeps trigger type - manual (0), data-driven (1) or scheduled(2).
schedulekeeps corresponding cron pattern for jobs that have been scheduled to a certain time.
pull_type is set to
schedule value is
NULL. In case you would like to schedule an artifact's execution, change the trigger type to
scheduled and set up the appropriate cron pattern. When the cron pattern is invalid or doesn't set, the UI throws an error and asks for valid input.
The scheduler process reads data from the database at the application's startup only. All other changes made to jobs triggering settings in the Datameer UI or via REST API calls are handled directly by the scheduler demon.
In case for some reason (e.g. incorrect UPDATE statement executed against Datameer database) the
pull_type for an artifact is set to
scheduled (2), but cron pattern remains
NULL or is invalid, at the next restart Datameer will report the below error and none of scheduled jobs will be triggered, even those which have the correct settings.
[anonymous] FATAL [2017-10-11 15:58:54.646] [pool-2-thread-2] (ContextListener.java:81) - uncaught exception from thread pool-2-thread-2
it.sauronsoftware.cron4j.InvalidPatternException: invalid pattern "2 y * * *". Error parsing hours field: invalid field "y", invalid element "y", invalid range, invalid value "y", invalid integer value.
[anonymous] FATAL [2017-10-10 17:18:32.952] [pool-2-thread-2] (ContextListener.java:81) - uncaught exception from thread pool-2-thread-2 it.sauronsoftware.cron4j.InvalidPatternException: invalid pattern: "NULL"
In order to fix this issue, it is required to review the artifacts configuration and ensure that scheduled jobs don't have invalid or NULL values configured.
The below MySQL query returns a list of scheduled jobs that have NULL for cron pattern.
SELECT pull_type,schedule,id FROM dap_job_configuration WHERE pull_type = 2 AND schedule = 'NULL';
In case this query returns any records, it's needed to change
pull_type for these jobs to
manual or set a valid cron pattern.
To let the scheduler demon pick up recent changes, please restart Datameer and ensure that no
InvalidPatternException appears in conductor log.