Job scheduling in Windows Azure, this has always been an interesting topic with plenty of solutions to get the “job” done: Mobile Services Scheduler, Quartz Scheduler, FluentScheduler, WebBackgrounder, ScheduledEnqueueTimeUtc, … While you can use these options to handle the job scheduling in your application, they’re not really targeting your Windows Azure applications running in a Windows Azure Web/Worker Role or Web Site.
That’s why – almost a year ago – Aditi announced a new scalable task scheduler which was built to target your applications running in Windows Azure, the Aditi Scheduler (built by one of the Windows Azure legends; Ryan Dunn). A few months ago they even celebrated 500.000 jobs and support for Windows Azure Storage Queues.
But a few days ago Microsoft announced a viable alternative to the Aditi Scheduler, the Windows Azure Scheduler:
Windows Azure Scheduler allows you to invoke actions – such as calling HTTP/S endpoints or posting a message to a storage queue – on any schedule. With Scheduler, you create jobs in the cloud that reliably call services both inside and outside of Windows Azure and run those jobs on demand, on a regularly recurring schedule, or designate them for a future date.
Let’s take a look…
Activating the preview feature
The Windows Azure Scheduler is still a preview feature so you’ll need to sign up for it first: http://www.windowsazure.com/en-us/services/preview/
At the moment there’s not support for the Windows Azure Scheduler in the management portal. The only way to interact with the scheduler is through the REST API or through the SDK. The Scheduler API is described in detail on MSDN: Scheduler REST API Reference.
Installing the SDK
The Scheduler SDK is available as a NuGet package (currently in preview) and is a part of the new management library which was released 2 weeks ago. To get started you’ll need to install this preview package:
PM> Install-Package Microsoft.WindowsAzure.Management.Scheduler -Pre
Getting started
In order to do some quick testing I’m using a publish settings file (which you can download here). The following class will parse that file and turn it into a CertificateCloudCredentials object, which you can use to authenticate with the management library:
public static class CertificateCloudCredentialsFactory
{
public static CertificateCloudCredentials FromPublishSettingsFile(string path, string subscriptionName)
{
var profile = XDocument.Load(path);
var subscriptionId = profile.Descendants("Subscription")
.First(element => element.Attribute("Name").Value == subscriptionName)
.Attribute("Id").Value;
var certificate = new X509Certificate2(
Convert.FromBase64String(profile.Descendants("PublishProfile").Descendants("Subscription").Single().Attribute("ManagementCertificate").Value));
return new CertificateCloudCredentials(subscriptionId, certificate);
}
}
And using this class is very easy:
var publishSettingsFilePath = @"D:\\azdem.publishsettings";
var subscriptionName = "Azdem194D92901Y";
var credentials = CertificateCloudCredentialsFactory
.FromPublishSettingsFile(publishSettingsFilePath, subscriptionName);
The Scheduler Service “runs” in a Cloud Service, that’s why we start by creating a Cloud Service. Just one thing you need to look out for is that you create the Cloud Service in a region which is supported by the Scheduler (support is limited to specific regions during the preview, more about that later).
var cloudServiceClient = new CloudServiceManagementClient(credentials);
var result = cloudServiceClient.CloudServices.Create("sandrino-cs1", new CloudServiceCreateParameters()
{
Description = "sandrino-cs1",
GeoRegion = "north europe",
Label = "sandrino-cs1"
});
Console.WriteLine(result.Status);
Console.WriteLine(result.HttpStatusCode);
Registering the Scheduler Resource Provider
We know that a Cloud Service can contain Virtual Machines (IaaS) or Web/Worker Roles (PaaS). But from what we can see now a Cloud Service can also host a “Resource Provider” (like the Scheduler). In order to use such a Resource Provider on your Cloud Service you need to enable that RP on your subscription first. If you don’t, you’ll see the following error:
An unhandled exception of type ‘Microsoft.WindowsAzure.CloudException’ occurred in Microsoft.WindowsAzure.Management.Scheduler.dll
Additional information: ForbiddenError: The subscription is not entitled to use the resource
Now go ahead and register the scheduler (this will happen for the complete subscription):
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.RegisterResourceProvider();
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
And after registering the resource provider we’re also able to query the properties:
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.GetResourceProviderProperties();
foreach (var prop in result.Properties)
{
Console.WriteLine(prop.Key + ": " + prop.Value);
}
Console.ReadLine();
For the scheduler this allows us to see the available plans and the supported regions:
Job Collections
The next thing we’ll do is create a “Job Collection” which is a container holding your jobs and applying quotas. This is also where you’ll choose between the free and the paid tier:
A job collection contains a group of jobs, and maintains settings, quotas and throttles that are shared by jobs within the collection. A job collection is created by a subscription owner, and groups jobs together based on usage or application boundaries. It’s constrained to one region. It also allows the enforcement of quotas to constrain the usage of all jobs in that collection; the quotas include MaxJobs and MaxRecurrence.
More information regarding the quotas that apply to both tiers can be found here: http://msdn.microsoft.com/en-us/library/windowsazure/dn479786.aspx
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.JobCollections.Create("sandrino-cs2", "jobcoll001", new JobCollectionCreateParameters()
{
Label = "jobcoll001",
IntrinsicSettings = new JobCollectionIntrinsicSettings()
{
Plan = JobCollectionPlan.Standard,
Quota = new JobCollectionQuota()
{
MaxJobCount = 100,
MaxJobOccurrence = 100,
MaxRecurrence = new JobCollectionMaxRecurrence()
{
Frequency = JobCollectionRecurrenceFrequency.Minute,
Interval = 1
}
}
}
});
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
HTTP(S) and Storage Queue Jobs
Now that we have our job collection in place we can start creating an actual job. The scheduler supports 3 types of jobs: HTTP, HTTPS and Storage Queues. Let’s take a look at the http support first:
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = new JobAction()
{
Type = JobActionType.Http,
Request = new JobHttpRequest()
{
Body = "customer=sandrino&command=sendnewsletter",
Headers = new Dictionary()
{
{ "Content-Type", "application/x-www-form-urlencoded" },
{ "x-something", "value123" }
},
Method = "POST",
Uri = new Uri("http://postcatcher.in/catchers/527af9acfe325802000001cb")
}
},
StartTime = DateTime.UtcNow,
Recurrence = new JobRecurrence()
{
Frequency = JobRecurrenceFrequency.Minute,
Interval = 1,
Count = 5
}
});
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
This job will be created in my Job Collection “jobcoll001″ and will send a POST request with a body and a custom header. Note that I specify the content type of the request, this is required for the Http(s) jobs. Since I’m using the paid plan in my JobCollection I’m allowed to run a job every minute. For this test I limited it to 5 executions.
Finally, if you look at the Uri I’m using http://postcatcher.in. This is a free service which lets you debug POST requests, which is perfect for what we’re doing here. Let’s take a look at what the Scheduler is doing:
As you can see it’s sending out the body I requested and you’ll see a bunch of headers, including the one I defined (x-something). The additional headers contain information about where the job was executed and for which Job Collection it happened.
{
"connection": "close",
"content-length": "40",
"content-type": "application/x-www-form-urlencoded",
"host": "postcatcher.in",
"x-forwarded-for": "137.116.241.137",
"x-ms-client-request-id": "988c7a64-55e1-41e4-8cf0-ce1eeca240ac",
"x-ms-execution-tag": "0726fa245447c91674c75db3f3564d63",
"x-ms-scheduler-execution-region": "North Europe",
"x-ms-scheduler-expected-execution-time": "2013-11-07T02:39:27",
"x-ms-scheduler-jobcollectionid": "jobcoll001",
"x-ms-scheduler-jobid": "7ce6971c-5aa1-4701-b6bd-02f63ee82d17",
"x-real-ip": "137.116.241.137",
"x-request-start": "1383791968800",
"x-something": "value123"
}
Now let’s change the action to Storage Queues:
var storageAccount = new CloudStorageAccount(new StorageCredentials("labdrino", ""), true);
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference("scheduled-tasks");
queue.CreateIfNotExists();
var perm = new QueuePermissions();
var policy = new SharedAccessQueuePolicy { SharedAccessExpiryTime = DateTime.MaxValue, Permissions = SharedAccessQueuePermissions.Add };
perm.SharedAccessPolicies.Add("jobcoll001policy", policy);
queue.SetPermissions(perm);
var sas = queue.GetSharedAccessSignature(new SharedAccessQueuePolicy(), "jobcoll001policy");
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = new JobAction()
{
Type = JobActionType.StorageQueue,
QueueMessage = new JobQueueMessage()
{
Message = "hello there!",
QueueName = "scheduled-tasks",
SasToken = sas,
StorageAccountName = "labdrino"
}
},
StartTime = DateTime.UtcNow,
Recurrence = new JobRecurrence()
{
Frequency = JobRecurrenceFrequency.Minute,
Interval = 1,
Count = 5
}
});
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
As you can see there’s a little more work to do when you work with Storage Queues. You’ll need to create a queue first and for that queue you’ll need to create a policy with the Add permission. For that policy you can then create a Shared Access Signature which will then be used by the Scheduler to post a message to the queue.
The result is a message in the queue which contains some information about the job together with our “hello there!” message:
Job History
Once your jobs are on the loose you want to have a way to follow up with what’s happening in the scheduler. This is possible by calling the GetHistory method with the Job Id. After creating a Job you’ll find the Job Id in the response, or you can just iterate of all jobs in the collection by calling the List method:
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
foreach (var job in schedulerClient.Jobs.List(new JobListParameters() { State = JobState.Enabled }))
{
Console.WriteLine("Job: {0} - Action: {1} - State: {2} - Status: {3}", job.Id, job.Action, job.State, job.Status);
foreach (var history in schedulerClient.Jobs.GetHistory(job.Id, new JobGetHistoryParameters()))
{
Console.WriteLine(" > {0} - {1}: {2}", history.StartTime, history.EndTime, history.Message);
}
}
Console.ReadLine();
And here is the output of this code:
Job: 34851054-f576-48b8-8c77-73b62b502022 – Action: Microsoft.WindowsAzure.Scheduler.Models.JobAction – State: Faulted – Status: Microsoft.WindowsAzure.Scheduler.Models.JobStatus
> 7/11/2013 2:52:18 – 7/11/2013 2:52:19: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 2:52:48 – 7/11/2013 2:52:50: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 2:53:19 – 7/11/2013 2:53:19: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 2:53:48 – 7/11/2013 2:53:50: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 2:54:20 – 7/11/2013 2:54:20: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 3:05:19 – 7/11/2013 3:05:19: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 3:05:49 – 7/11/2013 3:05:49: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queue
> 7/11/2013 3:06:18 – 7/11/2013 3:06:19: StorageQueue Action – The provided queue: ‘scheduled-tasks’ does not exist or the Sas Token does not have permission to add a message to the given queueJob: 4db6da21-af4a-4703-b988-671cbb6d5fd5 – Action: Microsoft.WindowsAzure.Scheduler.Models.JobAction – State: Completed – Status: Microsoft.WindowsAzure.Scheduler.Models.JobStatus
> 7/11/2013 2:32:13 – 7/11/2013 2:32:15: Http Action – Response from host ‘postcatcher.in’: ‘Created’ Response Headers: Connection: keep-alive
X-Response-Time: 6ms
Date: Thu, 07 Nov 2013 02:32:14 GMT
Set-Cookie: connect.sid=8SxhjZXandfZQc158Ng2tiYs.kyW9OSZGymzcIJW1eTJJ2MIACyhSyK6mfHVVqqj2r0E; path=/; expires=Thu, 07 Nov 2013 06:32:14 GMT; httpOnly
Server: nginx
X-Powered-By: Express
Body: Created
> 7/11/2013 2:33:14 – 7/11/2013 2:33:15: Http Action – Response from host ‘postcatcher.in’: ‘Created’ Response Headers: Connection: keep-alive
X-Response-Time: 18ms
Date: Thu, 07 Nov 2013 02:33:15 GMT
Set-Cookie: connect.sid=BJYkjeu3m26wBfr6G2SDgXZl.nhXEo24T3AVHEMYe4xJIm7gjDmhZvj69edIv4bui%2Bzs; path=/; expires=Thu, 07 Nov 2013 06:33:15 GMT; httpOnly
Server: nginx
X-Powered-By: Express
Body: Created
The history will give me some interesting information about the execution of the job. If something goes wrong with the job, this is where you’ll start to troubleshoot the issue.
Retries
Ok let’s say you’re sending an Http request to your website and the website is unavailable (a bug, a deployment, …). In that case you might want to retry that job after a few seconds. Well, this is something you can specify on action level with a retry policy:
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = new JobAction()
{
Type = JobActionType.Http,
Request = ...,
RetryPolicy = new RetryPolicy()
{
RetryCount = 5,
RetryInterval = TimeSpan.FromMinutes(1),
RetryType = RetryType.Fixed
}
},
StartTime = DateTime.UtcNow,
Recurrence = ...
});
In this example I’ll have a maximum of 5 retries with a 1 minute pause between the retries.
Dealing With Errors
Now if something goes wrong you might want to get an alert like a message being sent to a different queue (in a different datacenter), or by calling a different Url. This is possible by specifying an Error action on the Action. In this example I’m setting up a job with a StorageQueue but my SasToken is wrong. This will prevent the Scheduler from posting the message to the queue and will cause the ErrorAction to execute (which is a post to my postcatcher.in page with the “somethingiswrong” message.
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = new JobAction()
{
Type = JobActionType.StorageQueue,
QueueMessage = new JobQueueMessage()
{
Message = "hello there!",
QueueName = "scheduled-tasks",
SasToken = "not working",
StorageAccountName = "labdrino"
},
ErrorAction = new JobErrorAction()
{
Type = JobActionType.Http,
Request = new JobHttpRequest()
{
Uri = new Uri("http://postcatcher.in/catchers/527b0b75fe325802000002b6"),
Body = "type=somethingiswrong",
Headers = new Dictionary()
{
{ "Content-Type", "application/x-www-form-urlencoded" },
{ "x-something", "value123" }
},
Method = "POST"
}
}
},
StartTime = DateTime.UtcNow,
Recurrence = new JobRecurrence()
{
Frequency = JobRecurrenceFrequency.Minute,
Interval = 1,
Count = 5
}
});
Console.WriteLine(result.RequestId);
Console.WriteLine(result.StatusCode);
Console.ReadLine();
And here you can see the error message in postcatcher (the header also contain all the info you need regarding your job):
Recurrence
The scheduler also allows you to configure different types of recurrence, like saying that you want to run a job every day and this a maximum of 10 times (the Count property):
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = ...,
Recurrence = new JobRecurrence()
{
Frequency = JobRecurrenceFrequency.Day,
Interval = 1,
Count = 10
}
});
You could also specify that you want to run a job every day until a specific date, let’s say the end of the year:
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = ...,
Recurrence = new JobRecurrence()
{
Frequency = JobRecurrenceFrequency.Day,
Interval = 1,
EndTime = new DateTime(2013, 12, 31)
}
});
But you can also specify more complex schedules. Take a newsletter for example, you probably want to send out a weekly newsletter. In that case I can create a schedule that will execute a request every Monday at 11 AM UTC time:
var schedulerClient = new SchedulerClient(credentials, "sandrino-cs2", "jobcoll001");
var result = schedulerClient.Jobs.Create(new JobCreateParameters()
{
Action = new JobAction()
{
Type = JobActionType.Http,
Request = new JobHttpRequest()
{
Body = "customers=Europe-West",
Headers = new Dictionary()
{
{ "Content-Type", "application/x-www-form-urlencoded" },
},
Method = "POST",
Uri = new Uri("http://postcatcher.in/catchers/527af9acfe325802000001cb")
}
},
StartTime = DateTime.UtcNow,
Recurrence = new JobRecurrence()
{
// Frequency = JobRecurrenceFrequency.None,
Schedule = new JobRecurrenceSchedule()
{
Days = new List() { JobScheduleDay.Monday },
Hours = new List() { 9 },
Minutes = new List() { 11 }
}
}
});
Note: There seems to be a small issue with the Management library which does not allow us to change the Frequency to None (or Schedule). Because of this you won’t be able to create Jobs with this kind of schedule at the moment (you’ll get an error that the frequency Minute/Day/… is not supported when using a schedule).
Conclusion
With an easy to use api, support for storage queues, retry policies and error actions the Windows Azure Scheduler is a perfect way to handle job scheduling in your application. This will open up lots of new possibilities like scheduling jobs in a Windows Azure Web Site (without having to run a Worker Role), sending work to your Worker Roles, using the scheduler in your PHP, Node.js, … applications, building your own Windows Azure SQL Database Agent, …
More information:
- Scheduler homepage: http://www.windowsazure.com/en-us/services/scheduler
- Pricing details: http://www.windowsazure.com/en-us/pricing/details/scheduler (don’t forget to read the FAQ)
Enjoy!