Did you know that your computing tasks can be used as heating ? That’s the goal of the french company named Qarnot who provide computing heater.
A-MA-ZING idea !
Those tasks can be financial computing, Blender rendering, neural network treatment, image processing, …
How it works ?
Basically, you put your ressource (file, scripts, …) in a disk, run a task, get the result in the disk. Simple. The beauty of Qarnot is the ability to work with Docker image (custom or not) with an extreme facility. We will take a look at that later.
https://computing.qarnot.com/developers/overview/qarnot-computing-basics © QarnotTo setup all the process, you can do everything you want with the REST API or the SDK (using Python or Node.js).
Our first task : image processing
To test all that stuff, we will create a little program in .NET Core (<3) using a Docker image and the Python SDK. Our program will apply some filters to an image and returns the result in a new image.
The .NET Core project
Open Visual Studio 2017 and create a new .NET Core Console project called Rendering. Add the Nuget package SixLabors.ImageSharp :
PM> Install-Package SixLabors.ImageSharp -Version 1.0.0-beta0002
Then replace the code in Program.cs by this one :
using SixLabors.ImageSharp; namespace Rendering { class Program { static void Main(string[] args) { using (Image<Rgba32> image = Image.Load(args[0])) { image.Mutate(x =&gt; x .OilPaint() .Grayscale()); image.Save($"new_{args[0]}"); } } } }
Publish your app by right-clicking on the project and select Publish…
The Python script :
First step, install Qarnot by following that guide. Note : Clone the repository before starting everything !
When you are ready, create a new file named test.py with that script :
#!/usr/bin/env python import sys import qarnot # Build the full command line dotnet_cmd = 'dotnet Rendering.dll test.jpg' # Create a connection, from which all other objects will be derived conn = qarnot.Connection('qarnot.conf') # Create a task. The 'with' statement ensures that the task will be # deleted in the end, to prevent tasks from continuing to run after # a Ctrl-C for instance task = conn.create_task('processing-image', 'docker-batch', 1) # Store if an error happened during the process error_happened = False try: # Set the command to run when launching the container, by overriding a # constant. # Task constants are the main way of controlling a task's behaviour task.constants['DOCKER_REPO'] = 'microsoft/dotnet' task.constants['DOCKER_TAG'] = 'latest' task.constants['DOCKER_CMD'] = dotnet_cmd # Create a resource disk and add our input file. input_disk = conn.create_disk('processing-image-resource') input_disk.add_directory("bin") input_disk.add_file("test.jpg") # Attach the disk to the task task.resources.append(input_disk) # Submit the task to the Api, that will launch it on the cluster task.submit() # Wait for the task to be finished, and monitor the progress of its # deployment last_state = '' done = False while not done: if task.state != last_state: last_state = task.state print("** {}".format(last_state)) # Wait for the task to complete, with a timeout of 5 seconds. # This will return True as soon as the task is complete, or False # after the timeout. done = task.wait(60) # Display fresh stdout / stderr sys.stdout.write(task.fresh_stdout()) sys.stderr.write(task.fresh_stderr()) if task.state == 'Failure': # Display errors on failure print("** Errors: %s" % task.errors[0]) error_happened = True else: # Or download the results task.download_results('.') finally: task.delete(purge_resources=True, purge_results=True) # Exit code in case of error if error_happened: sys.exit(1)
Well, few things to explain :
# Create a connection, from which all other objects will be derived conn = qarnot.Connection('qarnot.conf')
You need to create the ‘qarnot.conf’ file to store your Token API.
[cluster] # url of the REST API url=https://api.qarnot.com [client] # auth string of the client token=__YOUR_TOKEN_API__
Here is my folders structure, where bin contains the .NET Core console app and qarnot the pyhon SDK. And yes, test.jpg is the image you want to process ;)
It’s time to execute our script and show the result !
$> python test.py ** Submitted ** FullyDispatched 0> Tue Feb 27 09:02:48 CET 2018 $>
Qarnot provide a monitoring portal that allow you to show all your tasks and disks.
At the end of the execution, you now have a new image named new_test.jpg in your root folder.