Did you know that your computing tasks can be used as heating ? That’s the goal of the french company named Qarnot who provide computing heater.
A-MA-ZING idea !
Those tasks can be financial computing, Blender rendering, neural network treatment, image processing, …
How it works ?
Basically, you put your ressource (file, scripts, …) in a disk, run a task, get the result in the disk. Simple. The beauty of Qarnot is the ability to work with Docker image (custom or not) with an extreme facility. We will take a look at that later.
https://computing.qarnot.com/developers/overview/qarnot-computing-basics © QarnotTo setup all the process, you can do everything you want with the REST API or the SDK (using Python or Node.js).
Our first task : image processing
To test all that stuff, we will create a little program in .NET Core (<3) using a Docker image and the Python SDK. Our program will apply some filters to an image and returns the result in a new image.
The .NET Core project
Open Visual Studio 2017 and create a new .NET Core Console project called Rendering. Add the Nuget package SixLabors.ImageSharp :
PM> Install-Package SixLabors.ImageSharp -Version 1.0.0-beta0002
Then replace the code in Program.cs by this one :
using SixLabors.ImageSharp;
namespace Rendering
{
class Program
{
static void Main(string[] args)
{
using (Image<Rgba32> image = Image.Load(args[0]))
{
image.Mutate(x =&gt; x
.OilPaint()
.Grayscale());
image.Save($"new_{args[0]}");
}
}
}
}
Publish your app by right-clicking on the project and select Publish…
The Python script :
First step, install Qarnot by following that guide. Note : Clone the repository before starting everything !
When you are ready, create a new file named test.py with that script :
#!/usr/bin/env python
import sys
import qarnot
# Build the full command line
dotnet_cmd = 'dotnet Rendering.dll test.jpg'
# Create a connection, from which all other objects will be derived
conn = qarnot.Connection('qarnot.conf')
# Create a task. The 'with' statement ensures that the task will be
# deleted in the end, to prevent tasks from continuing to run after
# a Ctrl-C for instance
task = conn.create_task('processing-image', 'docker-batch', 1)
# Store if an error happened during the process
error_happened = False
try:
# Set the command to run when launching the container, by overriding a
# constant.
# Task constants are the main way of controlling a task's behaviour
task.constants['DOCKER_REPO'] = 'microsoft/dotnet'
task.constants['DOCKER_TAG'] = 'latest'
task.constants['DOCKER_CMD'] = dotnet_cmd
# Create a resource disk and add our input file.
input_disk = conn.create_disk('processing-image-resource')
input_disk.add_directory("bin")
input_disk.add_file("test.jpg")
# Attach the disk to the task
task.resources.append(input_disk)
# Submit the task to the Api, that will launch it on the cluster
task.submit()
# Wait for the task to be finished, and monitor the progress of its
# deployment
last_state = ''
done = False
while not done:
if task.state != last_state:
last_state = task.state
print("** {}".format(last_state))
# Wait for the task to complete, with a timeout of 5 seconds.
# This will return True as soon as the task is complete, or False
# after the timeout.
done = task.wait(60)
# Display fresh stdout / stderr
sys.stdout.write(task.fresh_stdout())
sys.stderr.write(task.fresh_stderr())
if task.state == 'Failure':
# Display errors on failure
print("** Errors: %s" % task.errors[0])
error_happened = True
else:
# Or download the results
task.download_results('.')
finally:
task.delete(purge_resources=True, purge_results=True)
# Exit code in case of error
if error_happened:
sys.exit(1)
Well, few things to explain :
# Create a connection, from which all other objects will be derived
conn = qarnot.Connection('qarnot.conf')
You need to create the ‘qarnot.conf’ file to store your Token API.
[cluster] # url of the REST API url=https://api.qarnot.com [client] # auth string of the client token=__YOUR_TOKEN_API__
Here is my folders structure, where bin contains the .NET Core console app and qarnot the pyhon SDK. And yes, test.jpg is the image you want to process ;)
It’s time to execute our script and show the result !
$> python test.py ** Submitted ** FullyDispatched 0> Tue Feb 27 09:02:48 CET 2018 $>
Qarnot provide a monitoring portal that allow you to show all your tasks and disks.
At the end of the execution, you now have a new image named new_test.jpg in your root folder.
