The parallelization engine

Since the parallelization is currently taking place on one central machine, it is absolutely vital that the complexity of the parallelization algorithms is good. Preferably the complexity should be better than $\mathcal{O}(n^2)$, for $n$ instructions. It can, of course, never be faster than $\mathcal{O}(n)$.

The algorithms that lie behind the parallelization are described thoroughly in Chapter 4.

Since this system is intended to be used for numerical computation, it is vital that we do not compromise the numerical properties the algorithms we execute rely on. That is, we cannot allow the parallelizer to change the algorithm or method itself in a way that would significantly change the outcome of the computation.

This is something that I have not treated thoroughly in this report, since it is still not completely clear to me what we can do and what we cannot.

For now, the parallelizer plays safe. It will never change the order in which the calculations that make up a result are executed. It therefore will never change the properties of the numerical algorithm it parallelizes.

There are other technicalities that come into play, if the system is run on a heterogenous cluster. The nodes that participate in the cluster may have different machine precision (!) However, since the user is still guaranteed that the precision with which some algorithm is computed is never less than the lowest machine precision present in the cluster, this does not seem to be much of a problem.

If code is run on such a heterogenous cluster, the apparent machine precision can actually change between different runs of the program. I don't think this is a problem, as long as the user's are aware of this. It is definitely not worth seeking to guarantee a uniform machine precision throughout the cluster. This would impose a computing overhead on some of the nodes, and since numerical computations per definition are inaccurate anyway, this seems like a completely bad idea.