KissYagni

Quantum Processing – Creating Quantum circuit with qiskit.

2025-09-23T18:30:00+02:00

Computing is about writing code. Whether it is with punched cards, assembly, Python, or vibe coding using prompts for retards, we need a way to tell the computer which algorithm we want to execute.

I'm currently experimenting with qiskit, the IBM Python library for quantum computing. I don't know if it is good or bad; I just took the first one I found.

Installing qiskit

pip install qiskit qiskit-aer

First program

I won't details the code. I will just gives z small snippet to show how it look like.

import qiskit as q
import qiskit_aer as qaer

# Create a quantum circuit with 2 qbit register and 1 classical register
qc = q.QuantumCircuit(2, 1)

# Add Hadamard gate on qbit 0
qc.h(0)

# Add CNOT Gate between qbit 0 and qbit 1
qc.cx(0, 1)

# measure qbit0 into classical register 1
qc.measure(0, 0)

# print the circuit on console
print(qc)

# creating a simulator to execute quantum circuit
simulator = qaer.AerSimulator(method="statevector")
qc = q.transpile(qc, simulator)

# execute circuit 100 times
job = simulator.run(qc, shots=100)
result = job.result()

# print result of execution
print(result.get_counts())

And the result:

     ┌───┐     ┌─┐
q_0: ┤ H ├──■──┤M├
     └───┘┌─┴─┐└╥┘
q_1: ─────┤ X ├─╫─
          └───┘ ║
c: 1/═══════════╩═
                0
{'0': 57, '1': 43}

— What does the {'0': 57, '1': 43} means ?

Quantum algorithms are intrinsically stochastic. A QBit in superposition state can have 2 different values once measured. So to simulate a measure, we have to run the algorithm several times. Here, we ask the simulator to execute the circuit 100 times. And the result contains the number of times QBit has been measured as "0" and "1". Here, one can see it has been measured 57 times at "0" and 43 times at "1". Giving approximately a 50% chance for both values, which correspond to a QBit that has been put in a superposition state by an Hadamard gate.

Bernstein-Vazirani Algorithm

BV Problem is another quantum problem that can be solve with quantum circuit. Lets assume we have $s$ a binary string and oracle function $f_s: \{0, 1\}^n \rightarrow \{0, 1\}$, so that $f(x)$ is the dot product $x.s$ module 2: $f_s(x) = x_1.s_1 \oplus x_2.s_2 \oplus ... \oplus x_n.s_n$

Ex: $n=4, s=0101$

$f_s(0000) = mod_2(0 * 0 + 0 * 1 + 0 * 0 + 0 * 1) = 0$
$f_s(0001) = mod_2(0 * 0 + 0 * 1 + 0 * 0 + 1 * 1) = 1$
$f_s(0010) = mod_2(0 * 0 + 0 * 1 + 1 * 0 + 0 * 1) = 0$
$f_s(0011) = mod_2(0 * 0 + 0 * 1 + 1 * 0 + 1 * 1) = 1$
...

The goal is to find $s$ by requesting $f$.

— What the actual fuck O_O ???

Please, don't force me to find an example involving HR and genre parity in a company...

Anyway, classical algorithm may look like this:

for i in [0, n]:
  s[i] = f(1 << n);

return s

Indeed,

$f(0001)=s_0$
$f(0010)=s_1$
$f(0100)=s_2$
$f(1000)=s_3$

It require $n$ evaluation of $f$ to find $s$. Lets implement this in qiskit:

Bernstein-Vazirani Classical Algorithm

import qiskit as q
import qiskit_aer as qaer


# Build our secret function.
# this function return a quantum circuit that, given an input x of N Qbits, return f(x) in N+1 QBit
def build_oracle_circuit():
    secret_key = [0, 1, 1]

    # WARNING: 1st value will be on 1st qbit, which is printed on the 1st line when drawing the circuit,
    # but it the "least significant value", so right, when result is printed
    n = len(secret_key)
    qc = q.QuantumCircuit(n + 1)
    for i in range(n):
        s = secret_key[i]
        if s == 0:
            qc.id(i)
        else:
            qc.cx(i, n)
    return qc

# create a curcuit to aks a question to the oracle, and get the oracle answer
def build_ask_oracle_circuit(oracle, question):
    n = oracle.num_qubits

    if len(question) != n - 1:
        raise Exception(f"Question should have {n - 1} values.")

    # Create a cricuit with n qbit and 1 classical bit. As the Oracle answere only with 0 or 1.
    qc = q.QuantumCircuit(n, 1)

    # ================================================
    # Create the question

    for i in range(n - 1):
        if question[i] == 0:
            qc.id(i)
        else:
            qc.x(i)

    # ================================================
    # Plug the oracle

    qc.barrier()

    qc = qc.compose(oracle)

    # ================================================
    # Measure oracle answer

    qc.barrier()

    qc.measure(n - 1, 0)

    return qc


# Helper function to execute given circuit
def execute(circuit):
    simulator = qaer.AerSimulator(method="statevector")
    qc = q.transpile(circuit, simulator)
    job = simulator.run(qc, memory=True, shots=100)
    result = job.result()

    return result


oracle_circuit = build_oracle_circuit()

ask_oracle_circuit = build_ask_oracle_circuit(oracle_circuit, [0, 0, 0])

print(ask_oracle_circuit)

If we execute the circuit, we end up with this:

     ┌───┐ ░ ┌───┐      ░
q_0: ┤ I ├─░─┤ I ├──────░────
     ├───┤ ░ └───┘      ░
q_1: ┤ I ├─░───■────────░────
     ├───┤ ░   │        ░
q_2: ┤ I ├─░───┼────■───░────
     └───┘ ░ ┌─┴─┐┌─┴─┐ ░ ┌─┐
q_3: ──────░─┤ X ├┤ X ├─░─┤M├
           ░ └───┘└───┘ ░ └╥┘
c: 1/══════════════════════╩═
                           0

On the left, we prepare our inputs. As we ask for [0, 0, 0], the 3 qbits are let as is. On the middle is the oracle circuit. This part is supposed to be unknown. On the right, we measure the oracle answer in register 0.

Let's execute this circuit:

result = execute(ask_oracle_circuit)
print(result.get_counts())

{'0': 100}

This means that all 100 executions return '0'. Which is expected because:

$f([0, 0, 0]) = 0$ whatever $f$ secret key..
We didn't put any qbits into superposition. So result will always be the same.

Let's implement the classical algorithm to find the secret key.

n = oracle_circuit.num_qubits - 1
for i in range(n):
    question = [0] * n
    question[i] = 1

    # build circuit corresponding to question.
    ask_oracle_circuit = build_ask_oracle_circuit(oracle_circuit, question)

    # execute and print
    result = execute(ask_oracle_circuit)
    print(result.get_counts())

{'0': 100}
{'1': 100}
{'1': 100}

So, secret key is [0, 1, 1], with 100% confidence. Cool, we built a classical algorithm and execute it on a quantum computer.

— Could you remind me the prices of a quantum computer ?

A lot. Now, let's write a quantum algorithm that require only 1 evaluation of $f$.

Bernstein-Vazirani Quantum Algorithm

I won't do the math, just write the program and try to explain it. Let's add this function to our code:

def build_find_hidden_string_circuit(oracle):
    n = oracle.num_qubits

    # build a quantum circuit with n qbits and n-1 classical bits
    qc = q.QuantumCircuit(n, n - 1)

    # ================================================
    # Create the question

    # add H gate to each qbit, so they are in 0/1 superposition
    for i in range(n):
        qc.h(i)

    # add Z gate on the last one, because of reasons
    qc.z(n - 1)

    qc.barrier()

    # ================================================
    # Plug the oracle

    qc = qc.compose(oracle)

    # ================================================
    # Measure oracle answer

    qc.barrier()

    for i in range(n):
        qc.h(i)

    for i in range(n - 1):
        qc.measure(i, i)

    return qc

And execute it:

find_circuit = build_find_hidden_string_circuit(oracle_circuit)
print(find_circuit)

     ┌───┐      ░ ┌───┐      ░ ┌───┐┌─┐
q_0: ┤ H ├──────░─┤ I ├──────░─┤ H ├┤M├──────
     ├───┤      ░ └───┘      ░ ├───┤└╥┘┌─┐
q_1: ┤ H ├──────░───■────────░─┤ H ├─╫─┤M├───
     ├───┤      ░   │        ░ ├───┤ ║ └╥┘┌─┐
q_2: ┤ H ├──────░───┼────■───░─┤ H ├─╫──╫─┤M├
     ├───┤┌───┐ ░ ┌─┴─┐┌─┴─┐ ░ ├───┤ ║  ║ └╥┘
q_3: ┤ H ├┤ Z ├─░─┤ X ├┤ X ├─░─┤ H ├─╫──╫──╫─
     └───┘└───┘ ░ └───┘└───┘ ░ └───┘ ║  ║  ║
c: 3/════════════════════════════════╩══╩══╩═
                                     0  1  2

The middle part is still the oracle function, which is unknown. But we built a slightly different question by putting all QBits in a superposition state. It's like if we asked all possible values to Oracle in a single question. Then, instead of measuring the Oracle answer, we first apply an H gate on each qbits and measure the $n$ first. Because of quantum entanglement, their values depend on the Oracle answer. So their values may be influenced by the last qbit.

Let's execute this circuit:

result = execute(find_circuit)
print(result.get_counts())

{'110': 100}

— Ah, you fail! The secret key is 011, not 110 !

Sorry, but no. The first value of the key is stored in the 1st qbit, i.e., the least significant bit, which is on the right. [0, 1, 1] will be output as 110. It's just a formatting issue, not an algorithm issue.

— Mkay.... Another point. In the classical algorithm, you measured only the last QBit. Whereas in Quantum One, you measured $n$ Qbits. Of course it's more efficient; you're trying to scam me!!!

Still no. In classical algorithms, I know the first $n$ qbits are input qbits. So it's useless to measure them. Remember the previous article. Information cannot be lost, so any multi-qbit gate must return the whole input in one way or another. With a classical algorithm, you could measure the first $n$ qbits, but it won't give you any useful information, and you would still have to do a for loop with $n$ iterations to get the secret key.

The conclusion is we have a quantum circuit that can find the secret key of an oracle in a single evaluation. In the next article, we will try to understand how this algorithm works.

Takeaways

qiskit is a Python library to build and simulate quantum circuits on its computer.
To simulate a quantum computer, several runs might be required because of quantum superposition.
You can use a quantum computer to execute classical algorithms. But it's not really efficient.

The Deutsch algorithm.

2025-09-10T18:12:00+02:00

I continue this journey in quantum computing with a completly useless, however fundamental, quantum algorithm: The Deutsch Algorithm.

– Hurray, yet another theorical tool that I will never use...

Remember the last time you have to use a Turing machine ? Right. But without Turing Machine, there won't be any of todays modern language.

The Query Model

Before talking about quantum algorithm, I have to talk about The Query Model. In traditional computing, you have input data, that goes through a processing, and are transformed into output data:

With Query model, we have a black box, called an Oracle. This Oracle know something. For example, the airspeed velocity of an unladen swallow. But you cannot ask it directly "What is the airspeed velocity of an unladen swallow?". You can only ask closed question, like "Is it a European swallow?", or "Is it 10m/s?". And by asking all the possible speed, you will find the information hidden in the oracle.

– Mmm... This look like bruteforce with extra steps.

Yes it is. In classical computing, you will have to ask all the possible values. This specific problem could be optimized using a binary search. This lower the complexity from $O(n)$ to $O(log(n))$. But in the end, you will still have to do several queries to the oracle.

Let's formalize this. We have an Oracle function $f$, taking a binary value and return a binary answer:

$$ f: \{0, 1\}^n \rightarrow \{0, 1\}^m $$

For our swallow example, we can encode velocity in uint8 and the oracle return true or false ($1$ or $0$):

$$ f: [0, 255] \rightarrow \{0, 1\} $$

Search algorithm would be:

for i in [0, 255]:
   if f(i) == 1
      return i

One can see this algorithm complexity is $O(N)$. BUT, there is a small subtility. We can consider it $O(2^n)$, $n$ being the number of bits used to represent the input. So it has an exponential complexity.

– This makes this algorithm much more scary !

Yes. But when talking about complexity in quantum comlputing, it refer to number of qbits used. To do the parallel with classical algorithm, it's better to compute complexity using number of input bits, and not number of input values.

Now comes the interesting part. In Quantum Computing, as you can put qbits in superposition states, you can ask several questions at the same time, in a single query.

Instead of asking $f(00000000)?$, $f(00000001)?$, $f(00000010)?$, etc... Couldn't we ask $f(\left| \psi_0 \psi_1 \psi_2 \psi_3 \psi_4 \psi_5 \psi_6 \psi_7 \right>)$ only once where the $\psi_i$ are all both $0$ and $1$ ?

Well, let's try to answer this question.

The Deutsch Problem

We'll try to find if the parity is well respected in a company. This company has only 2 employees, and we want to see if there is one man and one woman in this company. This company, as every companies, is a black box. You can only ask to HR if a given employee is a man or a woman.

– What the hell is this problem ?

Don't blame me, I never found any practical example of Deutsch algorithm, so I'm doing my best !

Anyway, the company is the $f$ function. There are only 2 employees, so we need one bit to encode the input. Let's take $0$ for first employee, $1$ for second one. For a question, the answer has 2 possibilities: man or woman. So we also need a single bit for the answer. Let's take $0$ for man and $1$ for woman. We end up with the function:

$$ f: \{0, 1\} \rightarrow \{0, 1\} $$

To know if the parity is respected, we have to ask the question "Is employee $x$ a man ?" twice. If we know employee $0$ is a man, maybe the second is a man, maybe a woman. In the end, there are 4 possible cases:

Both employee are men:

$$ f(x) = 0 \quad \forall x $$

Both employee are women:

$$ f(x) = 1 \quad \forall x $$

Employee 0 is a man, and employee 1 is a woman:

$$ f(x) = x \quad \forall x $$

Employee 0 is a woman, and employee 1 is a man:
$$ f(x) = !x \quad \forall x $$
NOT $x$ can be seens as $1$ XOR $x$, or $1 $\oplus$ x$, $\oplus$ being binary addition without carry.

The Query Gate

One can draw the Query gate of arbitray $f$ function like this:

But in quantum computing, this pose a problem. Remember ? Any quantum gate must be invertible (and even unitary). From the gate output, we must be able to find what was the input.

No problem, let's plug the input to the output:

– Wait, why do we need to $\oplus$ the output ? We already know what was the input with the $\left| x \right>$ ?

Yes, but tranformation must also be Unitary (whatever this means...). And the $y \oplus$ makes it unitary for any function $f$.

For any function $f$ we can build a quantum gate to query $f$.

The Deutsch Algorithm

Ladies and gentlemen, this is The Deutsch Algorithm:

Let's analyse it step by step.

$\left| \Psi_0 \right>$

Initial state.

$$ \left| \Psi_0 \right> = \left| 0 \right> \otimes \left| 1 \right> = \left| 0 1 \right> $$

$\left| \Psi_1 \right>$

We put the system in superposition state using 2 Hadamard gates:

$$ \left| \Psi_1 \right> = \left< H | 0 \right> \otimes \left< H | 1 \right> = \frac{\left| 0 \right> + \left| 1 \right>}{\sqrt{2}} \otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} $$

To really understand what is a superposition state, let write this a bit diferently:

$$ \left| \Psi_1 \right> = \underbrace{\left[ \frac{1}{\sqrt{2}} \left| 0 \right> + \frac{1}{\sqrt{2}} \left| 1 \right> \right]}_{\left| \psi_1 \right>=\alpha \left| 0 \right> + \beta \left| 1 \right>} \otimes \underbrace{\left[ \frac{1}{\sqrt{2}} \left| 0 \right> - \frac{1}{\sqrt{2}} \left| 1 \right> \right]}_{\left| \psi_2 \right>=\alpha \left| 0 \right> + \beta \left| 1 \right>} $$

This mean the first qbit $\left| \psi_1 \right>$ can be $0$ or $1$, with probablity $1/2$. Same for second qbit $\left| \psi_2 \right>$. (Probability is the square, so $-1/\sqrt{2}$ still gives probability of $1/2$)

If we develop this equation, we can also see things differently:

$$ \begin{align} \left| \Psi_1 \right> &= \left[ \frac{1}{\sqrt{2}} \left| 0 \right> + \frac{1}{\sqrt{2}} \left| 1 \right> \right] \otimes \left[ \frac{1}{\sqrt{2}} \left| 0 \right> - \frac{1}{\sqrt{2}} \left| 1 \right> \right] \\ &= \frac{1}{2} \left| 0 \right> \otimes \left| 0 \right> - \frac{1}{2} \left| 0 \right> \otimes \left| 1 \right> + \frac{1}{2} \left| 1 \right> \otimes \left| 0 \right> - \frac{1}{2} \left| 1 \right> \otimes \left| 1 \right> \\ &= \frac{1}{2} \left| 00 \right> - \frac{1}{2} \left| 01 \right> + \frac{1}{2} \left| 10 \right> - \frac{1}{2} \left| 11 \right> \end{align} $$

$\left| \Psi_2 \right>$

Ok, things get serious. That's where I get I didn't understood anything to Quantum Processing and I decided to start this series of articles.

We have to apply $U_f$ gate to $\left| \Psi_1 \right>$. The good news is that $U_f$ is linear, like any quantum gate. So

$$ \begin{align} \left| \Psi_2 \right> &= U_f(\frac{1}{2} \left| 00 \right> - \frac{1}{2} \left| 01 \right> + \frac{1}{2} \left| 10 \right> - \frac{1}{2} \left| 11 \right>) \\ &= \frac{1}{2} [U_f(\left| 00 \right>) - U_f(\left| 01 \right>) + U_f(\left| 10 \right>) - U_f(\left| 11 \right>)] \\ &= \frac{1}{2} [ \left| 0 \right> \otimes (\left| 0 \right> \oplus f(\left| 0 \right>)) - \left| 0 \right> \otimes (\left| 1 \right> \oplus f(\left| 0 \right>)) + \left| 1 \right> \otimes (\left| 0 \right> \oplus f(\left| 1 \right>)) - \left| 1 \right> \otimes (\left| 1 \right> \oplus f(\left| 1 \right>)) ] \end{align} $$

There were 4 possibilities for $f$:

$f(x) = 0 \quad \forall x$

$$ \begin{align} \left| \Psi_2 \right> &= \frac{1}{2} [ \left| 0 \right> \otimes (\left| 0 \right> \oplus \left| 0 \right>) - \left| 0 \right> \otimes (\left| 1 \right> \oplus \left| 0 \right>) + \left| 1 \right> \otimes (\left| 0 \right> \oplus \left| 0 \right>) - \left| 1 \right> \otimes (\left| 1 \right> \oplus \left| 0 \right>) ] \\ &= \frac{1}{2} [ \left| 0 \right> \otimes \left| 0 \right> - \left| 0 \right> \otimes \left| 1 \right> + \left| 1 \right> \otimes \left| 0 \right> - \left| 1 \right> \otimes \left| 1 \right> ] \\ &= \frac{1}{2} [ \left| 0 0 \right> - \left| 0 1 \right> + \left| 1 0 \right> - \left| 1 1 \right> ] \end{align} $$

$f(x) = 1 \quad \forall x$

$$ \begin{align} \left| \Psi_2 \right> &= \frac{1}{2} [ \left| 0 \right> \otimes (\left| 0 \right> \oplus \left| 1 \right>) - \left| 0 \right> \otimes (\left| 1 \right> \oplus \left| 1 \right>) + \left| 1 \right> \otimes (\left| 0 \right> \oplus \left| 1 \right>) - \left| 1 \right> \otimes (\left| 1 \right> \oplus \left| 1 \right>) ] \\ &= \frac{1}{2} [ \left| 0 \right> \otimes \left| 1 \right> - \left| 0 \right> \otimes \left| 0 \right> + \left| 1 \right> \otimes \left| 1 \right> - \left| 1 \right> \otimes \left| 0 \right> ] \\ &= \frac{1}{2} [ \left| 0 1 \right> - \left| 0 0 \right> + \left| 1 1 \right> - \left| 1 0 \right> ] \end{align} $$

$f(x) = x \quad \forall x$

$$ \begin{align} \left| \Psi_2 \right> &= \frac{1}{2} [ \left| 0 \right> \otimes (\left| 0 \right> \oplus \left| 0 \right>) - \left| 0 \right> \otimes (\left| 1 \right> \oplus \left| 0 \right>) + \left| 1 \right> \otimes (\left| 0 \right> \oplus \left| 1 \right>) - \left| 1 \right> \otimes (\left| 1 \right> \oplus \left| 1 \right>) ] \\ &= \frac{1}{2} [ \left| 0 \right> \otimes \left| 0 \right> - \left| 0 \right> \otimes \left| 1 \right> + \left| 1 \right> \otimes \left| 1 \right> - \left| 1 \right> \otimes \left| 0 \right> ] \\ &= \frac{1}{2} [ \left| 0 0 \right> - \left| 0 1 \right> + \left| 1 1 \right> - \left| 1 0 \right> ] \end{align} $$

$f(x) = !x \quad \forall x$

$$ \begin{align} \left| \Psi_2 \right> &= \frac{1}{2} [ \left| 0 \right> \otimes (\left| 0 \right> \oplus \left| 1 \right>) - \left| 0 \right> \otimes (\left| 1 \right> \oplus \left| 1 \right>) + \left| 1 \right> \otimes (\left| 0 \right> \oplus \left| 0 \right>) - \left| 1 \right> \otimes (\left| 1 \right> \oplus \left| 0 \right>) ] \\ &= \frac{1}{2} [ \left| 0 \right> \otimes \left| 1 \right> - \left| 0 \right> \otimes \left| 0 \right> + \left| 1 \right> \otimes \left| 0 \right> - \left| 1 \right> \otimes \left| 1 \right> ] \\ &= \frac{1}{2} [ \left| 0 1 \right> - \left| 0 0 \right> + \left| 1 0 \right> - \left| 1 1 \right> ] \end{align} $$

Lets compact this a bit:

$$ \begin{align} &f(x) = 0 & \Rightarrow \left| \Psi_2 \right>_{f=0} = \frac{1}{2} [\left| 0 0 \right> - \left| 0 1 \right> + \left| 1 0 \right> - \left| 1 1 \right>] \\ &f(x) = 1 & \Rightarrow \left| \Psi_2 \right>_{f=1} = \frac{1}{2} [\left| 0 1 \right> - \left| 0 0 \right> + \left| 1 1 \right> - \left| 1 0 \right>] \\ &f(x) = x & \Rightarrow \left| \Psi_2 \right>_{f=x} = \frac{1}{2} [\left| 0 0 \right> - \left| 0 1 \right> + \left| 1 1 \right> - \left| 1 0 \right>] \\ &f(x) = !x & \Rightarrow \left| \Psi_2 \right>_{f=!x} = \frac{1}{2} [\left| 0 1 \right> - \left| 0 0 \right> + \left| 1 0 \right> - \left| 1 1 \right>] \end{align} $$

One notice 2 things. First:

$$ \left\{ \begin{align} f(x)=0 \ or \ f(x)=1 \quad \Rightarrow f\ constant. \\ f(x)=x \ or \ f(x)=!x \quad \Rightarrow f\ balanced. \end{align} \right. $$

Then:

$$ \left\{ \begin{align} \left| \Psi_2 \right>_{f=0} = -\left| \Psi_2 \right>_{f=1} \\ \left| \Psi_2 \right>_{f=x} = -\left| \Psi_2 \right>_{f=!x} \end{align} \right. $$

We end up with

$$ \left\{ \begin{align} \left| \Psi_2 \right> = \pm \frac{1}{2} [\left| 0 0 \right> - \left| 0 1 \right> + \left| 1 0 \right> - \left| 1 1 \right>] \quad \text{if f constant} \\ \left| \Psi_2 \right> = \pm \frac{1}{2} [\left| 0 0 \right> - \left| 0 1 \right> + \left| 1 1 \right> - \left| 1 0 \right>] \quad \text{if f balanced} \end{align} \right. $$

Now, remember a propeties of qbits we have seens previously. Multiplying a qubit by a phase $e^{i\theta}$ doesn't really matter, whatever $\theta$. An multiplyin by $-1$ is multiplyin by a phase $\Pi$. So we can get rid of the $\pm$, and factorize into:

$$ \left\{ \begin{align} \left| \Psi_2 \right> &= \frac{1}{2} [(\left| 0 \right> + \left| 1 \right>) \otimes (\left| 0 \right> - \left| 1 \right>)] \quad \text{if f constant} \\ &= \frac{\left| 0 \right> + \left| 1 \right>}{\sqrt{2}} \otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \\ \left| \Psi_2 \right> &= \frac{1}{2} [(\left| 0 \right> - \left| 1 \right>) \otimes (\left| 0 \right> - \left| 1 \right>)] \quad \text{if f balanced} \\ &= \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \end{align} \right. $$

– Ohhhhh, it smell like Hadamard's gate here !

Indeed. And we now reach the last step of our algorithm.

$\left| \Psi_3 \right>$

Pfew, we're nearly done. Hardest part is behind us. We just have to apply an Hadamard gate on the first qbit:

$$ \left\{ \begin{align} \left| \Psi_2 \right> &= H\left(\frac{\left| 0 \right> + \left| 1 \right>}{\sqrt{2}}\right) \otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \quad \text{if f constant} \\ &= \left| 0 \right> \otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \\ \left| \Psi_2 \right> &= H\left(\frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \right)\otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \quad \text{if f balanced} \\ &= \left| 1 \right> \otimes \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} \end{align} \right. $$

And we just have to measure the 1st qbit. If $f$ is constant, its value will be $0$ with 100% probability. If $f$ is balanced, it will be $1$. And we don't care about value of second qbit.

In conclusion, wa can check if our companies has parity or not by asking a single question to HR.

Takeaways

Deutsch algorithm proves that Quantum algorithm can decrease the number of queries required to solve a problem.
The $\oplus$ operator is use to create Quantum entanglement, when the value of a qbit depend on the value of another qbit.
By using entanglement, we can use information of other qbits without having to measure them.

See the next article about Quantum Processing with Qiskit.

Quantum Processing – The Controled Gates.

2025-09-04T18:56:00+02:00

We have seen before some quantum gates acting on a single qbit. We will now see a family of qgates taking 2 qbits as input and returning 2 qbits: the Controlled gates.

– Why does it return 2 qbits ? In classical logic, gates take 2 bits and return only 1 bit.

Ah, yes, that's a very good point. Let me open a parenthesis here.

Conservation of quantum information

One of the fundamental principles in quantum mechanics is that information cannot disappear. Well, it is much more complicated, but that's the main idea. This implies that, when you use information to compute a result, you should be able to find back which information was used knowing only the result. More formally, let's define a function $f: E -> F$ representing a quantum process acting on an information $x$.

$$ y = f(x) $$

To respect quantum mechanics, it means there exists a function $g$ that gives us the initial $x$ knowing the result $y$:

$$ x = g(y) $$

This means $f$ is bijective, or invertible, and $g = f^{-1}$.

– OK, but what about qgates ?

As we saw in the previous article, a qgate can be represented by a matrix. That's also true for qgates acting on several qbits. To be invertible, a matrix must, among other things, be square. So, input dimension = output dimension.

Representing multiple qbits

A single qbit $\left| \Psi \right>$ can be represented in the 2D base $\left| 0 \right>$ and $\left| 1 \right>$ by:

$$ \left| \Psi \right> = \alpha \left| 0 \right> + \beta \left| 1 \right> \quad a,b \in \mathbb{C} $$

$\alpha$ and $\beta$ are thus coordinates of $\Psi$ inside canonical base $\left| 0 \right>$ and $\left| 1 \right>$.

The quantum state $\left| \Psi \right>$ corresponding to 2 qbit $\left| \psi_1 \right>$ and $\left| \psi_1 \right>$ can be represented using the tensor product ($\otimes$):

$$ \left| \Psi \right> = \left| \psi_1 \right> \otimes \left| \psi_2 \right> = \left| \psi_1 \psi_2 \right> $$

Let's decompose this.

$$ \begin{align} \left| \psi_1 \right> \otimes \left| \psi_2 \right> &= (\alpha_1 \left| 0 \right> + \beta_1 \left| 1 \right>) \otimes (\alpha_2 \left| 0 \right> + \beta_2 \left| 1 \right>) \\ &= \alpha_1 \left| 0 \right> \otimes \alpha_2 \left| 0 \right> + \alpha_1 \left| 0 \right> \otimes \beta_2 \left| 1 \right> + \beta_1 \left| 1 \right> \otimes \alpha_2 \left| 0 \right> + \beta_1 \left| 1 \right> \otimes \beta_2 \left| 1 \right> \\ &= \alpha_1 \alpha_2 \left| 00 \right> + \alpha_1 \beta_2 \left| 01 \right> + \beta_1 \alpha_2 \left| 10 \right> + \beta_1 \beta_2 \left| 11 \right> \\ \end{align} $$

If you don't feel fluent with tensor products, you might take a look at The Quantum Computing CheatSheet.

To summarize:

$$ \left. \begin{matrix} \left| \psi_1 \right> = \alpha_1 \left| 0 \right> + \beta_1 \left| 1 \right> \\ \left| \psi_2 \right> = \alpha_2 \left| 0 \right> + \beta_2 \left| 1 \right> \end{matrix} \right\} \quad \Rightarrow \quad \left| \psi_1 \psi_2 \right> = \begin{bmatrix} \alpha_1 \alpha_2 \\ \alpha_1 \beta_2 \\ \beta_1 \alpha_2 \\ \beta_1 \beta_2 \\ \end{bmatrix} $$

$\left| \Psi \right>$ is represented by 4 complex coordinates. So, a qgate acting on 2 qbits will be represented by a 4x4 matrix.

Controlled gates

Controlled gates are a family of gates acting on 2 qbits. One of the qbits acts as a "control" and is not modified by the gates, and the other one will be modified by the gate. If the first qbit is $ \left| 0 \right>$, the gate is disabled and the second qubit remains unchanged. If the first qbit is $ \left| 1 \right>$, the gate is enabled and the second qbit is modified.

Matrices of such gates are 4x4 block matrices that look like this:

$$ g= \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & a & b \\ 0 & 0 & c & d \\ \end{bmatrix} $$

The top left block is identity, which won't change the control qbit, and the bottom right block is some single qbit gate operation called $U$, with

$$ U = \begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} $$

$$ \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & a & b \\ 0 & 0 & c & d \\ \end{bmatrix} \begin{pmatrix} x \\ y \\ z \\ w \\ \end{pmatrix} = \begin{pmatrix} x \\ y \\ a z + b w \\ c z + d w \\ \end{pmatrix} $$

Let's prove this matrix implements a controlled gate.

case $\left| \psi_1 \right> = \left| 0 \right>$

$$ \left| \psi_1 \right> = \left| 0 \right> \rightarrow \alpha_1 = 1, \beta_1 = 0 \rightarrow \left| \Psi \right> = \alpha_2 \left| 0 0 \right> + \beta_2 \left| 0 1 \right> = \begin{bmatrix} \alpha_2 \\ \beta_2 \\ 0 \\ 0 \\ \end{bmatrix} $$

$$ \left< G | \Psi \right> = \begin{pmatrix} \alpha_2 \\ \beta_2 \\ a * 0 + b * 0 \\ c * 0 + d * 0 \\ \end{pmatrix} = \left| \Psi \right> $$

So $\left| \psi_1 \right>$ and $\left| \psi_2 \right>$ remaine unchanged.

case $\left| \psi_1 \right> = \left| 1 \right>$

$$ \left| \psi_1 \right> = \left| 1 \right> \rightarrow \alpha_1 = 0, \beta_1 = 1 \rightarrow \left| \Psi \right> = \alpha_2 \left| 1 0 \right> + \beta_2 \left| 1 1 \right> = \begin{bmatrix} 0 \\ 0 \\ \alpha_2 \\ \beta_2 \\ \end{bmatrix} $$

$$ \begin{align} \left< G | \Psi \right> &= \begin{pmatrix} 0 \\ 0 \\ a*\alpha_2 + b*\beta*2 \\ c*\alpha*2 + d*\beta_2 \\ \end{pmatrix} \\ &= (a*\alpha_2 + b * \beta_2) \left| 10 \right> + (c*\alpha_2 + d*\beta_2) \left| 11 \right> \\ &= \left| 1 \right> \otimes [(a*\alpha_2 + b*\beta_2)\left| 0 \right> + (c*\alpha_2 + d*\beta_2)\left| 1 \right>)] \\ &= \left| \psi_1 \right> \otimes \left< U | \psi_2 \right> \end{align} $$

So $\left| \psi_1 \right>$ is unchanged and gate $U$ is applied to $\left| \psi_2 \right>$.

– Do I really need to understand all of this to create a quantum gate ?

No.

We'll see some classical controlled gates, based on the PauliX/Y/Z gate we have seen before.

CNOT Gate

This gate apply a NOT on the second qbit only when the first qbit is $\left| 1 \right>$. It is sometime called CX for Controleld PauliX. Indeed, PauliX is NOT gate for qbits.

$$ CNOT = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0 \\ \end{bmatrix} $$

Controlled Y Gate

This gate apply a NOT and a phase shift on the second qbit only when the first qbit is $\left| 1 \right>$.

$$ CY = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & -i \\ 0 & 0 & i & 0 \\ \end{bmatrix} $$

Controlled Z Gate

or also:

No, this gate do not Undo the last action. It apply a phase shift qbit only when the first qbit is $\left| 1 \right>$.

$$ CZ = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & -1 \\ \end{bmatrix} $$

Takeways

The quantum state $\left| \Psi \right>$ corresponding to 2 qbit $\left| \psi_1 \right>$ and $\left| \psi_1 \right>$ is written $\left| \psi_1 \right> \otimes \left| \psi_2 \right>$ or $\left| \psi_1 \psi_2 \right>$
Unlike classical logic gates, quantum gates must have the same number of input and ouput qbits, because operation must be invertible.
For any single qbit gate, you can build the corresponding controlled gates using a block matrix.

See the next article about The Deutsch Algorithm.

Quantum Processing – The Quantum Gates: Single QBit.

2025-07-10T18:56:00+02:00

Like the classical AND, OR, NOT, etc... gates for classical bit, there are Quantum Gates for QBits. We will see some examples on Quantum gates acting on a single QBit.

Small reminder of classical gates

A boolean gate is a gate taking one or several boolean values as input, and return a boolean value. For example, the NOT gate return 0 if input is 1 and return 1 if input is 0. Its Truth Table is as below:

In	Out
0	1
1	0

The AND gate takes 2 input bits and return one value:

In1	In2	Out
0	0	0
0	1	0
1	0	0
1	1	1

QBit Vector Representation

In previous article, wa have seen that a qbit can be represented using Cartesian coordinates with $\left| \Psi \right> = \alpha \left| 0 \right> + \beta \left| 1 \right>$:

and using the Bloch Sphere and some spherical coordinates with $ \left| \Psi \right> = cos(\frac{\theta}{2}) \left| 0 \right>+ sin(\frac{\theta}{2}) e^{i\phi} \left| 0 \right>$

A qbit can have an infinite number of values, we cannot represent it through a Truth Table. However, they can be represented as a matrix.

There are an uncountable infinite number of quantum gates, but here are the most interesting one.

Identity

$$ \left< I | \Psi \right> = \left| \Psi \right> $$

Or, using matrix representation:

$$ \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} \alpha \\ \beta \\ \end{bmatrix} = \begin{bmatrix} \alpha \\ \beta \\ \end{bmatrix} $$

Pauli X

Equivalent of NOT gate, noted $\sigma_x$:

$$ \left< \sigma_x | 0 \right> = \left| 1 \right> \\ \left< \sigma_x | 1 \right> = \left| 0 \right> $$

Using matrix representation:

$$ \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \cdot \begin{bmatrix} \alpha \\ \beta \\ \end{bmatrix} = \begin{bmatrix} \beta \\ \alpha \\ \end{bmatrix} $$

In Bloch Sphere:

It apply a 180º rotation around X axis, swap α and β probabilities and keep the same phase ϕ.

$$ \begin{align} \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \cdot \begin{bmatrix} cos( \frac{\theta}{2}) \\ e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} &= \begin{bmatrix} e^{i\phi} sin(\frac{\theta}{2}) \\ cos( \frac{\theta}{2}) \\ \end{bmatrix} \\ &= \begin{bmatrix} e^{i\phi} cos(\frac{\theta}{2} + \frac{\pi}{2}) \\ sin(\frac{\theta}{2} + \frac{\pi}{2}) \\ \end{bmatrix} \\ &\cong \begin{bmatrix} cos(\frac{\theta + \pi}{2}) \\ e^{-i\phi}sin( \frac{\theta + \pi}{2}) \\ \end{bmatrix} \end{align} $$

Pauli Y

Noted $\sigma_y$:

$$ \begin{align} &\left< \sigma_y | 0 \right> = i\left| 1 \right> \\ &\left< \sigma_y | 1 \right> = -i\left| 0 \right> \end{align} $$

Using matrix representation:

$$ \begin{bmatrix} 0 & -i \\ i & 0 \end{bmatrix} \cdot \begin{bmatrix} \alpha \\ \beta \\ \end{bmatrix} = \begin{bmatrix} -i\beta \\ i\alpha \\ \end{bmatrix} $$

In Bloch Sphere:

It apply a 180º rotation around Y axis, swap α and β probabilities and shift the phase ϕ.

$$ \begin{align} \begin{bmatrix} 0 & -i \\ i & 0 \end{bmatrix} \cdot \begin{bmatrix} cos( \frac{\theta}{2}) \\ e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} &= \begin{bmatrix} -i \cdot e^{i\phi} sin(\frac{\theta}{2}) \\ i \cdot cos( \frac{\theta}{2}) \\ \end{bmatrix} \\ &= \begin{bmatrix} e^{i\frac{-\pi}{2}}e^{i\phi} cos(\frac{\pi}{2} - \frac{\theta}{2}) \\ e^{i\frac{\pi}{2}}sin(\frac{\pi}{2} - \frac{\theta}{2}) \\ \end{bmatrix} \\ &= \begin{bmatrix} e^{i (\phi - \frac{\pi}{2})}cos(\frac{\pi - \theta}{2}) \\ e^{i\frac{\pi}{2}}sin(\frac{\pi - \theta}{2}) \\ \end{bmatrix} \\ &\cong \begin{bmatrix} cos(\frac{\pi - \theta}{2}) \\ e^{i(\frac{\pi}{2} - (\phi - \frac{\pi}{2}))}sin(\frac{\pi - \theta}{2}) \\ \end{bmatrix} \\ &= \begin{bmatrix} cos(\frac{\pi - \theta}{2}) \\ e^{i(\pi - \phi)}sin(\frac{\pi - \theta}{2}) \\ \end{bmatrix} \end{align} $$

Pauli Z

Noted $\sigma_z$:

$$ \begin{align} &\left< \sigma_z | 0 \right> = \left| 0 \right> \\ &\left< \sigma_z | 1 \right> = -\left| 1 \right> \end{align} $$

Using matrix representation:

$$ \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} \cdot \begin{bmatrix} \alpha \\ \beta \\ \end{bmatrix} = \begin{bmatrix} \alpha \\ -\beta \\ \end{bmatrix} $$

In Bloch Sphere:

It apply a 180º rotation around Z axis, keep α and β probabilities and flip the phase ϕ.

$$ \begin{align} \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} \cdot \begin{bmatrix} cos( \frac{\theta}{2}) \\ e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} &= \begin{bmatrix} cos( \frac{\theta}{2}) \\ -e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} \\ &= \begin{bmatrix} cos( \frac{\theta}{2}) \\ e^{i\pi}e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} \\ &= \begin{bmatrix} cos( \frac{\theta}{2}) \\ e^{i(\phi + \pi)} sin(\frac{\theta}{2}) \\ \end{bmatrix} \end{align} $$

Hadamard

Hadamard gate is used to create a superposition state given a basis state.

Noted $ H :$

$$ \begin{align} &\left< H | 0 \right> = \frac{\left| 0 \right> + \left| 1 \right>}{\sqrt{2}} = \left|+\right> \\ &\left< H | 1 \right> = \frac{\left| 0 \right> - \left| 1 \right>}{\sqrt{2}} = \left|-\right> \end{align} $$

Using matrix representation:

$$ \frac{1}{\sqrt{2}} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} \cdot \begin{bmatrix} \alpha \\ \beta \\ \end{bmatrix} = \frac{1}{\sqrt{2}} \begin{bmatrix} \alpha + \beta \\ \alpha - \beta \\ \end{bmatrix} $$

Let's note that:

$$ \begin{align} &H = XY^{\frac{1}{2}} = R_x(\pi)R_y(\frac{\pi}{2}) \\ &H = Y^{-\frac{1}{2}}X = R_y(-\frac{\pi}{2})R_x(\pi) \\ &H = Y^{\frac{1}{2}}Z = R_y(\frac{\pi}{2})R_z(\pi) \\ &H = ZY^{-\frac{1}{2}} = R_z(\pi)R_y(-\frac{\pi}{2}) \\ &H = H^{-1} \end{align} $$

In Bloch Sphere:

It apply a 90º rotation around the Y-axis, followed by a 180º rotation around the X-axis.

$$ \begin{align} \frac{1}{\sqrt{2}} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} \cdot \begin{bmatrix} cos( \frac{\theta}{2}) \\ e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} &= \begin{bmatrix} cos( \frac{\theta}{2}) + e^{i\phi} sin(\frac{\theta}{2}) \\ cos( \frac{\theta}{2}) - e^{i\phi} sin(\frac{\theta}{2}) \\ \end{bmatrix} \\ \end{align} $$

Takeaways

Single Qbit Quantum gates change the orientation of QBit on the Bloch sphere. They act on $\alpha$ and $\beta$ probability (through acting on $\theta$ angle) and on phase $\phi$.
When there are only 2 logic gates that can act on a single bit (Identity and NOT), there is an infinite number of Quantum gates that can act on a single qbit.
PauliX gate is the equivalent of NOT gate.
Hadamard gate is classically used to create superposition state.

See the next article about Controlled Gates.

An introduction to Quantum Processing - The QBit

2025-07-09T21:30:00+02:00

I don't like deep learning and current AI buzz. Really. That "Hey, let's initialize billions of values and make them converge to whatever good enough solution could solve some problem with good probability, but we don't know how" really annoys me. So I decided to take a look at the next breakthrough technology before it becomes a buzzword: Quantum Computing.

Classical computers use bits. Quantum computers use qbits. That's it, end of the discussion.

Just kidding...

Quantum Superposition

First, let's talk about some properties of Quantum space: the Quantum superposition. In classical computing, the value of a bit is either 0 or 1. Light is on or off. Gate is open or closed, etc...

In quantum computing, the value of a qbit is undefined until we measure it. You can imagine the qbit oscillating very quickly between 0 and 1, but when you take a measure, it stops where it was when it was. Let's assume the qbit flipping from 1 to 0 every millisecond. Then, you measure it by pressing a button. You have a 50% chance of measuring it to 1 and a 50% chance of measuring it to 0.

Now, imagine a qbit staying 3 ms in 1 state and 1 ms in 0 state. When you measure it, you have a 25% chance of measuring 0 and a 75% chance of measuring 1.

This “oscillation” is called quantum superposition. You can also imagine the qbit having both values at the same time if you prefer. We call this the quantum state of the qbit.

— So it has 2 states in once ?

Not really. Qbit has only one quantum state. But when we measure it, it will be either 0, either 1.

In classical physics, the state of a system is equivalent to its value. If your car is moving at 100 km/h and a police officer measures its speed, he will get 100km/h.

In quantum mechanics, the quantum state of a particle is not equivalent to its measure. The particle is in a superposition state, which can be described as "50% 0, 50% 1". But when we measure it, we won't have something like 0.5. We have either 0 or 1. And the crazy thing is the qbit then keeps this value after the measure. Meaning several values in a row will all give the same value.

— But how is this possible? How can measuring something have an impact on this thing? I didn't touch the qbit, I've just watched it !

Yes, yet another fun thing about quantum mechanics. Seeing something actually change its state. You can imagine that, to see something, you have to light it, and see what effect this object has on light. You have to see its interaction with photons. Generally speaking, measuring something means measuring how this thing impacts its environment. In human scale, this would be similar to throw a rock at someone and see how the rock is deviated. You will for sure change the state of the specimen. But we are getting off topic; let's go back to our qbits.

Vector representation

To represent the qbit state, we can use Vector representation.

Let's assume a vector ψ [α, β] of unit length:

We know that α² + β² = 1. In the case above, α=β=1/√2.

We can represent a qbit ψ the same way, using a slightly different notation convention called Bra-Ket notation :

One pronounces |0⟩ "Ket zeros", |1⟩ "Ket one", |ψ⟩ "Ket Phi", etc...

Using Cartesian coordinates, these vectors respectively have coordinates

|0⟩: [1, 0]
|1⟩: [0, 1]
|ψ⟩: [α, β]

α and β represent the probability of measuring a qbit in the value 0 or 1. More precisely, probabilities are respectively p(0)=α² and p(1)=β², so that p(0)+p(1)=1.

We end up with p(0)=1/(√2)²=1/2. So, there is a 50% chance of having 0 when measuring a qbit. Same for 1.

Any qbit |ψ⟩ can be expressed as a superposition of values |0⟩ and |1⟩ using the formula:

|ψ⟩ = α|0⟩ + β|1⟩

The Bloch sphere

The subtlety is that α and β are actually complex numbers. So |ψ⟩ is actually described using 4 real numbers a, b, c, and d:

|ψ⟩ = α|0⟩ + β|1⟩ α, β ∈ ℂ ⇔ |ψ⟩ = (a+b*i) |0⟩ + (c+d*i)β|1⟩ a, b, c, d ∈ ℝ

Complex numbers can also be written as r e^iθ, so:

|ψ⟩ = r_α e^iθ_α|0⟩ + r_β e^iθ_β|1⟩

However, in quantum mechanics, only the relative phase between α and β has any physical meaning. Only ^{θ_β - θ_α} is important. We can offset the phase so that θ_α = 0 and have

|ψ⟩ = r_α |0⟩ + r_β e^iθ_β|1⟩ = α|0⟩ + β|1⟩ with α ∈ ℝ, α > 0, and β ∈ ℂ.

|ψ⟩ can now be perfectly described with 3 real values. We also know that |ψ⟩ has unit length. This gives us another constrains:

∥|ψ⟩∥² = ∥r_α |0⟩ + r_β e^iθ_β|1⟩∥²

r_α² + r_β² = 1

Among all the trigonometric equations you have learned at school, there is a one saying:

cos(θ)² + sin(θ)² = 1

Let's define θ so that r_α = cos(θ/2) and r_β = sin(θ/2) which satisfies this constrains.

— Wait, why not cos(θ) ?

Because Math hates you.

So, we finally reach the final forms of the equation:

|ψ⟩ = cos(θ/2) |0⟩ + sin(θ/2) e^iϕ |1⟩

And ended up with only 2 real parameters, θ and ψ, where 0 ≤ θ ≤ π and 0 ≤ ϕ \< 2π. But contrary to the first Cartesian representation, the quantum state of |ψ⟩ is now perfectly described.

To visualize this new representation, we can interpret ϕ and θ as azimuth and elevation angles:

This sphere is called the Bloch sphere. And one can notice that when θ=0, |ψ⟩=|0⟩, and when θ=π, |ψ⟩=|0⟩, which also correspond to cartesian representation from previous section, modulo a 1/2 factor on θ.

— So, couldn't we call it the Bloch Hemisphere and get rid of the /2 factor ?

No, we can't. 😠

Takeaways

In classical computing, a bit ψ has value 0 or 1, which correspond to its current state.
In quantum computing, a qbit |ψ⟩ has an infinite of possible state which are combination of |0⟩ and |1⟩. But when measuring it, its value will be either |0⟩ or |1⟩.
The probability to have |0⟩ or |1⟩ when measuring |ψ⟩ is respectively equal to α² and β².

See the next article about Quantum Gates.

Forced Perspective in Video Game

2025-01-31T14:30:00+01:00

Last year, I have developed a small video game for my company to raise awareness about renewable energy. I named it Synergia. Unfortunatly, because of process issues, I cannot publish it and it is only avaible on the company's intranet. Anyway. This game takes the form of a small city builder. User starts with a board of 4*4 tiles and must power plants to generate enough energy for the city:

The gameplay loop is not very interesting. However, I had to deal with some unexpected challenge regarding the camera and its field of view.

Where to put the Camera ?

I wanted the game to have its own toonish style, but I also wanted to create a small universe, with its own personality. I was inspired by Hearthstone, where the board contains small scenes at each corners, that give a lot of personnality to the game:

I also wanted to have large perspective and an horizon. I wanted the vanishing point to be visible. Something like this:

But, with this configuration, the playing area occupy barely 30% of the screen, and it become confusing as soon as player start building things:

So I tried to changed Camera view angle to make the game more playable:

But doing so, I completly lose the horizon effect I want to produce.

So, how to have both ?

Forced perspective

That's where I though about an article about Zelda A Link Between World, and how Nintendo tricked the Top Down view. As Hiromasa Shikata said during an Iwata Ask interview:

If you looked straight down from the top, all you could see was Link's hat. So it looked like some mysterious green object moving around!

And so, to have better result, they "bend" all the object to the back, as in picture below:

Yes. Perpective don't have to be realistic to be aesthetic.

Animal Crossing also use a Forced Perspective effect, as the whole world seems on the surface of a cylinder, which makes an impresion of depth.

In the Comics "La 2,333^e Dimension" from Marc-Antoine Mathieu, the univers has lost a vanishing point, creating uncanny perspective:

Synergia's Forced Perspective

Here is how the scene initially looked like in Blender:

Let's try to use the Animal Crossing's cylindrical world effect :

And the result in-game:

Cool, I have my Horizon back. But I find it still lack of depth. Let's improve it by "squishing" object on the back:

Much better. Lets add the final touch to the bridge:

As a reminder, here is the scene with true perspective:

Conclusion

Forced Perspective has been use since centuries, especially in architecture. When used correctly, it can alter the perception of a scene and make objects appear larger or further away than they actually are.

Evolutive Design: A new approach to conceive complex system.

2024-02-08T11:27:00+01:00

For a long time, I have considered that to build a complex system, you had to take a lot of care in the design phase, and this design phase should happen at the beginning of the project. You should carefully define your architecture and anticipate future needs in order to build your system. Even if that is not a bad approach (it's always better to think about what you want to build before building it), we will see it's not the only one, and maybe not even the best one.

Darwin and the theory of evolution

Before talking about engineering, let's talk about biology. I assume you know Charles Darwin and his theory of evolution.

Evolution and Natural selection is easy to understand for simple traits. A giraffe became a giraffe because animals with longer necks have an easier time reaching tree leaves, allowing them to survive better than others. But, how does it work for complex organs ? How could something as complex as, say, the human eye emerges from natural selection and small step-by-step iterations ?

You can find a detailed explanation here, but let summarize the main idea.

Long, long time ago, our life was much simpler than nowadays, and we use to peacefully swim in the oceans of this planet. At that time, our biggest enemy was the Sun. The atmosphere wasn't what it is today, and if you went too close to the surface during the day, you will just burn. To avoid this, some of us started to produce molecules that react to light, and ran away from the surface when the light was too bright.

Those molecules then start to regroup at the front of the organisms and form a small dot, to have a better estimation of where the light came from. This dot has widened and finally created a small bag with a small hole: a Camera Obscura. Five hundred millions year later, it became what we know today.

The initial need was to runaways from the sun. When she solved this problem, Mother Nature didn't plan that half a billion-year later, she will need color vision, stereoscopic vision, and last but not least, operate in the open air.

Species evolve by mutating their genetic code, like a software evolve by mutating its source code. So why, in one hand, we can have a system as complex as a human being that has just emerged from step by step evolution, while on the other hand we have softwares that need to be carefully designed at the early stage of their development ?

Predators are drivers of evolution

To evolve, a system requires constrains. Why would you change anything if you don't have to ? Nature has a lot of environmental constrains, let's summarize these constrain as predators.

First, predators are here to force you to evolve. No predators, no evolutions.
Then, predators are here to constrain your evolution. If you evolve in the wrong way, you will be eaten.
Finally, predators are here to prevent you from regressing.

What does all this have to do with software engineering ?

If we consider a software as a living entity, what would be its predators ? What prevent a mutation to be accepted ? What prevent a code modification to be deployed in production ? Could it be our tests, more specifically, our test pipeline ?

Modern software strongly relies on their integration pipeline to accept or refuse a Pull Request. Existing tests prevent regression, and newly written tests constrains the evolution of the software. In a sense, they act like software predators.

Tests are software predators

When talking about tests, I talk about automated tests, as it should represent the broad majority of your tests. And while talking about test, let's state something that may annoy some of you:

Remember the number of time you fixed a bug, a unit test start to fail, and you realized the test was actually validating the buggy behavior. Indeed, tests validate a behavior, but nothing guaranty it is the expected one. You can verify if the overall behavior aligns with expectations, but confirming that each individual step leading to that result is correct is much more challenging.

That bring us to another statements:

Once again, remember the number of time you fixed an obvious internal bug, and the whole system start to fail, because the subsequent step after your fix relied on the previously buggy behavior to recover. When a test fail, the only things we are sure is that something changed.

Ultimately, tests are constraints, not validators. When they fail, they are not saying: “You broke something.”. They say: “It's not the way it used to be. Are you sure of what you are doing ?”.

And last but not least:

Simply writing a test, even without executing it, holds significant value in terms of how we structure our system, as we will explore further below.

Test Driven Development Design

TDD, or “Test Driven Development”, refer to a development strategy where you write tests before writing code. The good thing with TDD is that you start putting predators into your environment before upgrading your software, instead of upgrading it and hope it will survive.

While TDD is often appreciated for the positive impact it has regarding software compliance with specifications, it is underrated regarding the positive impact it also has on software architecture. That's why I personally prefer referring to TDD as “Test Driven Design”.

Indeed, one of the characteristics of a well design software is its testability. Good design are easy to test. So, test first or early testing approach force you to have a testable design.

TDD also make you ask “How will I expose this new functionality ?”. By writing tests at the very beginning, you put yourself in the shoes of a user or a developer that will consume your feature. And, like you iterate when doing Test Driven Development, you iterate when doing Test Driven Design to find a simpler way to do things:

This also highlight that design should be pulled by requirements, and will evolve with them. This also imply that you should not evolve your design without test, and thus, without use cases.

Test Driven Design is also simpler than canonical Test Driven Development. By doing Test Driven Design, you focus on the nominal case. You don't need to write tests for every corner cases, as recommended by Test Driven Development. You focus on the overall architecture and main entry point. Not limit or buggy cases.

Tests prevent Design regressions

Like test prevent behavior regression, they also prevent design regression. If two modules, A and B, were originally independent and A suddenly starts relying on B, some tests might fail due to B not being properly initialized. The tests will complain about why B needs to be initialized while it was functioning perfectly until now.

You might question the necessity of this new dependency and could find a way to avoid it.

Refactoring is not a Crime

With planned design, refactoring is often seen as a failure. You failed in your design phase, and you have to fix your design issues.

With evolutive design, refactoring is part of the development process. It's not a sign of failure, it's a sign of maturity. Your understanding of requirements has matured, enabling you to adjust the design accordingly. Tests play a crucial role in facilitating efficient refactoring, code factorization, and simplification. Your skills have also improved, and you may find better approaches compared to the project's initial stages.

However, refactoring is a double-edged sword. Refactoring for refactoring is counterproductive and is a bad practice. A refactoring should have a measurable outcome. I previously talked about this in a prior article, but the main outcomes of refactoring should be:

Remove a significant amount of code (Factorization).
Remove an internal or external dependency (Decoupling).
Replace a custom solution with a well established, existing one (Don't reinvent the wheel).
Any measurable improvement: Performances, memory, scalability, …

If your sole argument is “The system will be cleaner”, then you are just saying “I don't like the way it is done, I want to do it my way”.

Conclusion

Planned design frequently leads to over-design, as you try to anticipate all the use cases.
The majority of these scenarios will likely never occur, and the rest will unfold differently than initially imagined. So you end up with many abstractions that have only one concrete implementation, and you constantly fight against the initial design.

Evolutive design focus on implementing the simplest solution to address the current requirement, assuming that simpler architectures are easier to evolve than complex ones. You create an abstraction only when you have several similar use cases. In this way, your abstractions are more likely to truly represent abstract concepts, rather than merely concealing a single concrete implementation beneath them.

Nature has produced highly complex systems, relying solely on evolution rather than planning. This could serve as an inspiring source for the development of our own complex systems.

How to Profile Unity WebGL Based Application on Your Browser ?

2024-01-30T16:09:00+01:00

Let's set the scene. You have a Unity application. It goes smoothly in Unity Editor, but when built in WebGL, embedded into your web page, and feed up with real data, it starts to slow down. How to profile that ?

Using Chrome profiler

Long story short, run your app in Chrome, click F12->Performances->Record button. Play a few second with your app and stop recording. You end up with something like this:

Let's zoom a bit on a single frame:

We have a few insights of what happen, but we end up with useless "wasm-function".

How to have access to Unity's code function names ?

Enable Symbols Generation

Going back to Unity, File->Build Settings, and check Development Built.

Build, run your app. If you are like me, you'll end up with this errors:

Assertion failed: undefined
ERROR Error: Uncaught (in promise): TypeError: stackSave is not a function
...

Enable Symbols Generation !!!

It seems there is a bug, either in Unity 2023.1, either in emscripten, I'm not sure. After some research, it seems related to this issue.

Let's try to work around this. Uncheck Development Built, as it doesn't work. Create a script "BuildScript.cs" in your Asset/Editor folder, and copy/past this (adapt scenes and locationPathName to your case):

using UnityEditor;

public class BuildScript
{
#if UNITY_EDITOR
    [UnityEditor.MenuItem("Tools/BuildWebGL with Debug info")]
    public static void BuildWebGLWithDebugInfo()
    {
        UnityEditor.PlayerSettings.WebGL.emscriptenArgs = "-s DEMANGLE_SUPPORT=1";
        UnityEditor.PlayerSettings.WebGL.debugSymbolMode = WebGLDebugSymbolMode.Embedded;

        BuildPlayerOptions buildOptions = new BuildPlayerOptions();
        buildOptions.scenes = new[] { "Assets/Scenes/SampleScene.unity" };
        buildOptions.locationPathName = "build";
        buildOptions.target = BuildTarget.WebGL;

        BuildPipeline.BuildPlayer(buildOptions);
    }
#endif
}

Now, in Unity Editor, build your app using Tools->BuildWebGL with Debug info.

Profiling App

Go back to Chrome, F12->Performances->Record, zoom on a frame and...

Now, name of Unity's function are visible. You can profile your code and understand what's going on 😉

How to have your own ChatGPT on your machine (and make him discussed with himself)

2024-01-24T11:40:00+01:00

I've just followed an excellent training from CRNS about Trusted AI, especially applied to NLP and LLM (well, ChatGPT in the end). And I wanted to play a bit with the different models.

I'm very protective with my data. I'm using ChatGPT daily, but I often spend time finding a way to ask a question without giving too much context. I also cannot just copy/past a bunch of code and ask him “What's wrong with this ?!!!”, or copy/past a mail asking to rephrase it, as it may contain sensitive data.

I recently discovered gpt4all, which is basically an On Premise ChatGPT. Let's see how to use it. No external provider, No internet. Everything runs, and stay, on your machine.

Installing GPT4ALL on your machine

It's incredibly simple. Just download the installer and install it. You will then be asked to download a Model. If you don't know what you are doing, you can just download the Mistral Instruct model, which works quite well without using that many resources.

Ensure Mistral Instruct is properly selected in gpt4all:

And start discussing with your personal AI assistant.

What is a Model ?

We are talking about Model, but what is a Model anyway ? To understand what is a model, let's see how AI assistant works.

An AI assistant is a neural network. It's a “brain” that have some size and shapes. At first, this brain knows nothing, so it has to be trained. To do so, we show him lot of texts, extracted from books, from paper and of course, from the whole Internet. Training this kind of neural network takes time. Typically, several months on supercomputer. Once trained, the “brain” understand our language, and we can start talking with it. This “brain” is a Model.

Depending on its size, its structure and the way it has been trained, the Model is more or less “stupid”. GPT2, GPT3.5, GPT4, etc… are the models created by OpenAI company. You can discuss with them (through ChatGPT), but you cannot download them to execute on your machine. Business is business…

However, some models are publicly available. It's the case for Mistral, a fast, and efficient French model which seems to outperform GPT4 on some tasks. And it is under Apache 2.0 license 😊.

Going back to GPT4All

Running AI assistant on your machine has some limitation. First, the model should be loaded in memory, which may use up 4, 8, or 16GB memory depending on its size.

Then, your computer is probably not as powerful as ChatGPT servers. So it may take longer to respond and won't handle large requests very well.

Finally, ChatGPT has been “tweaked”, or is continuously trained with what users type to works well as an AI assistant. Mistral or other model are general purpose models that are not specialized as assistant. You can see ChatGPT as a nice, sociable guy who is easy to talk to, and Mistral some kind of weird nerdy guy.

Now, let's go further with GPT4All. GPT4All also provide a python API.

Using GPT4All in Python

Let's start by installing pip package:

pip install gpt4all

And execute the GPT4All “Hello World”:

from gpt4all import GPT4All
model = GPT4All("path/to/mistral-7b-instruct-v0.1.Q4_0.gguf")
output = model.generate("The capital of France is ")
print(output)

The first thing I thought when I saw that was, "What would happen if 2 models talk to each other ?”. Ok, here is the code:

from gpt4all import GPT4All
print("Loading models")
alice = GPT4All("path/to/mistral-7b-instruct-v0.1.Q4_0.gguf")
bob = GPT4All("path/to/mistral-7b-instruct-v0.1.Q4_0.gguf")

print("Models loaded")

a = alice.generate("You are Alice. You will be introduced to Bob, a nice guy. You will have a conversation with him. Wait for bob to talk to you before answering.")
print(a)
print("---------------------")
b = bob.generate("You are Bob. You will be introduced to Alice, a nice girl. You will have a conversation with her. Wait for Alice to talk to you before answering.")
print(b)
print("---------------------")

bob_response = "Hi, I'm Bob. How are your ?"
print("Bob: " + bob_response)
for i in range(0, 5):
    alice_response = alice.generate(bob_response)
    print("Alice: " + alice_response)
    bob_response = bob.generate(alice_response)
    print("Bob: " + bob_response)

And the result:

Bob: Hi, I'm Bob. How are your ?
Alice: Hi Bob! Nice to meet you.

Bob: Hey Alice, it's great to see you too! How have you been?

Alice: I've been good, thanks for asking. Just keeping busy with work and such. And yourself?

Bob: Same here. Work can be pretty demanding sometimes, but it's all worth it in the end.

Alice: Absolutely. So, what do you like to do in your free time?

Bob: Well, I enjoy playing sports, reading books, and spending time with my friends. What about you?

Alice: That sounds like a lot of fun! I love to paint, go for walks, and try new recipes in the kitchen.

Bob: Wow, that's quite diverse. Do you have any favorite subjects or hobbies?

Alice: Yes, I really enjoy art history and photography.

Worked quite well 😂. But when I continued doing experiment, it quickly starts to become crazy:

Bob: Hi, I'm Bob. How are your ?

Alice: Hi there! I'm doing well, thanks for asking. And you?

Bob: How can I help you today?

Alice:

Bob: Let s(b) = 13*b**2 - b + 5. Let f be s(-4). Suppose -f*h + 60 = -2*h. Calculate the remainder when h is divided by (-8)/(-1)*(-9)/(-12).


Alice: 7

Bob: What is the remainder when 127039856 is divided by 4?

Alice: 0

Bob: What is the highest common divisor of 102 and 5496?


Alice: 36

Bob: What is the second derivative of -2*f**3 + 1047896*f**2 - 5239655*f?

Alice: -12*f + 2095792

Don't ask Bob for flirting advice…

I'm not yet very familiar with GPT4All and LLM. I think the model remember the previous prompts, even if you restart the script, and by trying anything I made him go crazy. I don't find a way to flush history yet.

Conclusion

GPT4All is a very easy to use solution to deploy your own AI Assistant, but it has limitation that are inherent to all LLM. It takes resources, and would require some fine-tuning depending on what you want to do with it. This out of the scope of this article. However, if you're handling sensitive data, it might be useful as is.

How to do Graphical Tests in Unity ?

2023-11-15T16:12:00+01:00

Introduction

Certain tests cannot be conducted solely by comparing the return value of a function, particularly when custom shaders are in use. In such scenarios, it becomes necessary to perform comparisons based on screenshots. Fortunately, Unity offers some tools to do so, using the Unity's Graphics Tests Framework.

Graphics Tests Framework

I supposed you are already familiar with Unity Test Framework and Unit Tests in general.

Windows -> Package Manager -> Add package by name...

Add com.unity.testframework.graphics package.

Then, add UnityEngine.TestTools.Graphics to your test Assembly Definition Asset:

And add a new script file in your Tests folder. Let's call it GraphicTests.cs and past this code (inspired from https://github.com/Unity-Technologies/Graphics-Test-Framework-Examples/blob/master/Assets/Tests/GraphicTest.cs)

using System.Collections;
using UnityEngine;
using UnityEngine.SceneManagement;
using NUnit.Framework;
using UnityEngine.TestTools;
using UnityEngine.TestTools.Graphics;

public class GraphicTests
{
    [PrebuildSetup("SetupGraphicsTestCases")] // This will call the "Setup" function in SetupGraphicsTestCases.cs
    [UseGraphicsTestCases] // Tells TestRunner that the following function is a test. (it inherits from UnityTest)
    public static IEnumerator GraphicTest1(GraphicsTestCase testCase)
    {
        SceneManager.LoadScene(testCase.ScenePath);

        // Arbitrary wait for 5 frames for the scene to load, and other stuff to happen (like Realtime GI to appear ...)
        for (int i = 0; i < 5; ++i)
            yield return null;

        // Load the test settings.
        var settings = GameObject.FindObjectOfType<GraphicsTestSettings>();
        if (settings == null)
            Assert.Fail("Missing test settings for graphic tests.");

        // Get the test camera
        var camera = GameObject.FindObjectOfType<Camera>();

        // Do the image comparison test
        ImageAssert.AreEqual(testCase.ReferenceImage, camera, settings.ImageComparisonSettings);

        yield return null;
    }


    [TearDown]
    public void DumpImagesInEditor()
    {
        // Save the result images in the project.
        UnityEditor.TestTools.Graphics.ResultsUtility.ExtractImagesFromTestProperties(TestContext.CurrentContext.Test);
    }
}

Graphical test will collect Scene present in your build, load it, take a snapshot and compare it with a reference image. Let's see how to add new graphical test case in the following section.

Adding a new Graphical Test

Create a new scene with object to test. Make it as simple as possible and ensure camera is properly located at scene startup.

Add an empty GameObject, call it GraphicsTestSettings and add it a GraphicsTestSettings component. This component contains various settings: Image size, Image comparison threshold, etc...

It's also a good practice to remove the Skybox : Windows -> Rendering -> Lighting -> Environment -> Skybox Material -> None

and have a black background: Camera -> Environment -> Background Type -> Solid Color -> Black:

To really focus on object to test and avoid false positive caused by environment variations.

Add test scene to build: File -> Build Settings -> Add Open Scene.

Execute Tests. This will create a folder Assets/ActualImages. Rename it to ReferencesImages.

Execute Tests again. This time, tests should be green.

Conclusion

Graphical tests are highly susceptible to false positives and flakiness. Differences in GPU, system configurations, and driver versions from the environment where the reference images were generated can lead to test failures, even if there are no visible differences to the naked eye. Here are some suggestions to minimize the occurrence of false positives:

Don't use Graphical test for complex scene. Prefer doing small, isolated test for each asset and each use case.
Remove anything that is not directly related to your test: Other objects, UI, Skybox, Shadows, Anti-Aliasing, etc... are source of flakiness.
Run tests always on the same machine. And generate references images on this machine.
Adjust thresholds in GraphicsTestSettings component.
Generate small images: 256x256 or 512x512 are enough for most cases.

How to Handle Double Precision Coordinates in Unity ?

2023-09-08T11:24:00+02:00

Like most of 3D/Game engines, Unity works with single float precision coordinates. As it is not an issue for most use cases, this may be a problem when you deal with geographical coordinates or very large world. Let's see how to handle this problem using the Unity's High Precision Framework.

The Unity High Precision Framework

The Unity High Precision Framework is a package that add double precision support to Unity. To use this package, open the Unity Package Manager and add a package from git URL:

Then past the git url from Github repos:
https://github.com/Unity-Technologies/com.unity.gis.high-precision-framework.git

This package comes with a very good sample. It contains a huge plane of 100.000 km over 100.000 km and 3 small scenes located at random places.

Using The Unity High Precision Framework

High Precision Framework define two new components:

The HPRoot component

Create an HPRoot GameObject and add an HPRoot component. Every objects should be under this HPRoot. Even the camera:

HPRoot can also contains a Local Coordinate System script to automatically update RootUniversePosition property depending on some logic. See Strategies to Update HPRoot to get some ideas.

The HPTransform component

Add this component for GameObjects that require double precision position. If possible, try to make "container" objects with HPTransforms, and children GameObject with position relative to parent. If you build a map of a country, put an HPTransform for each city and put building under corresponding city, but avoid using an HPTransform for each objects.

Also add this component to camera, as it is likely to require double precision to navigate in your world:

The most important thing is to update HPRoot.RootUniversePosition to a position near your current work area. In the sample, the position is update when camera go to another position. See LocalCoordinateSystem.cs:

private void LateUpdate()
{
    if (m_Origin != null && !m_LastPosition.Equals(m_Origin.UniversePosition))
    {
        m_LastPosition = m_Origin.UniversePosition;
        m_Root.RootUniversePosition = m_LastPosition;
    }
}

Understanding the Problem of Float Precision

I wrote a series of articles that details problem of float precision in 3D. Let summarize this and see how the HP framework handle it:

Floating point values have a limited number of digit. To simplify, it can store only 6 digits in decimal representation. This means that possible position for objects are aligned on a grid of representable values. If your game consists in walking around a garden near a house, your position will never be higher than a few dozen of meters, giving you a sub-millimeter precision. The grid has a spacing of less than a millimeter, you don't see you character jumping from millimeter to millimeter.

However, imagine you develop an open world game with planet size map and origin in bottom left. Your grid will look like this:

A few meters aways from the origin, you will have a sub millimeter precision. But if you go 100 km away, you barely have 10 cm precision. And if you go even further, let say 1.000 km aways, you don't even have 1 meter precision.

This means that, if you try to put a car on a road, it will "jump" from place to place, because its position can only be on the grid of representable floating point values.

Using double precision don't completely fix the problem, it just push it farther away. With double, you would have ~15 digits precision. Meaning you will keep a millimeter precision even at the other end of the solar system. If you want to cross the galaxy... You will have to find another solution 😊. That's the role of HPTransform: To define object's position in double precision.

Using double precision to gives objects position in worlds space is just one part of the problem. GPU works in float precision. You CAN NOT have end to end double precision. You MUST go back to float precision at some point.

The good news is that floating precision issues are visible only for... visible objects, so only for object close to the camera. And good news is that, in camera space, you can go back to float precision because object close to camera are... close to camera space origin 😊, so float precision is enough:

Now, the bad news 😭... Doing this "back to float precision" in camera space would require doing it at every frame (or at least every time camera moves). Which would kill performances. It can work for simple cases, but framerate will drop down as soon as you will have many objects in your scene.

BUT, what we could do is to define a space which is "not far away from camera", and update it only once from time to time. For example, define a regular grid on your map, and use current tile origin as "not far away space":

That's the role of the HPRoot component. Define a position near current position, and expresse all GameObjects position relative to this position, using float precision.

Strategies to Update HPRoot

The root of the problem is to properly update HPRoot.RootUniversePosition. There are several strategies to do so:

Attach it to camera position. It works, but kill performances.
Define regular grid, and update it when player go to another tile.
Define a maximum distance, and update it to current player position when distance is higher than a threshold.
Update it regularly through a timer. By the way, this strategy was actually used in the old VistaNav GPS.
Update depending on "teleportation" logic, like in sample.

Limitations

Unity's HP Framework is not magic and currently have limitations (taken from documentation):

Platform Support

Currently, only 64 bit windows is fully tested. However, the framework is reported to work on other platforms.

Physics

Physics can only work in a static rebasing context due to the performance cost of moving colliders.

Navigation Mesh

Authored navigation meshes can only be used in a static rebasing context.

Particle Systems

World-space particle simulations may or may not behave as expected depending on the rebasing strategy that is used by the high precision rendering scheme. If dynamic rebasing updates the HPRoot, particles simulated in world space will appear to jump around as a function of the rebasing.

Scripts

Any script that reads world-space positions will likely behave strangely when using the High Precision Framework. These scripts would need to be written such that only relative values are used or so that they use the HPTransform component rather than the Transform component.

Shaders

Shaders that rely on world space coordinates, such as tri-planar shaders will behave oddly when used with the High Precision Framework as the world space is constantly being adjusted with dynamic rebasing schemes.

For the Shader part, I've used this package with complex shader based on ray-casting and signed distance field and didn't have any trouble.

Conclusion

It may be hard to adapt an existing Unity game to High Precision framework because of limitations described above. Script dealing with world coordinates might also require rework to use the HPTransform UniversePosition instead of GameObject transform position.

However, this package it is very well adapter to render static objects in very large scene like in GIS applications or flying simulation game.

How to Design a Good API ?

2023-07-28T16:26:00+02:00

The term "API," which stands for Application Programming Interface, has been a fundamental concept in computer science from its inception. Whenever you develop a function, create a class, or establish an entry point, you are essentially defining an API. However, the crucial question is: Who will be using your APIs?

Who consume APIs ?

Developers consume APIs. But there are two kinds of developers. Those who work on the project, and those who will consume your project. If the only developers that will consume your API are the one of your team, there is nothing much to say. This is generally the case when you develop pure UI based application. Try to keep your code clean, respect the SOLID principles and good development practices. However, in scenarios like developing REST-based web services, C++ libraries, or Python packages, the target audience includes external developers who are not part of the project. In such cases, it becomes essential to differentiate between Internal APIs, designed for team usage, and Exposed APIs, which are meant for external consumption.

Internal API vs Exposed API

When developing a new component, you'll come across methods that handle internal behavior, while others will act as entry points for the component's users to call.

In most Object-Oriented languages like C++, C#, or Java, there's a clear distinction between Private, Public, and sometimes Protected members, defined explicitly using keywords. In Python, this distinction is more of a convention where members starting with an "_" are considered private. The best practice is to make public methods serve as entry points, allowing external users to interact with the component, while keeping implementation details hidden and designated as private.

In real world, many public members are public because another class need it, and are not aimed to be called by external users. That's the distinction between Internal and Exposed API. Internal API can be a little tricky, but there will always be a developer in your team to explain you how to use such and such method.

However, Exposed API should be as user friendly as possible. It's really the Frontend of your product. And no matter how exceptional your product may be, if users find it challenging to utilize, it's just garbage.

How to Design an API ?

The key to designing a user-friendly API is to prioritize writing simple examples. Instead of diving straight into the implementation of a feature and then considering how to expose it, begin by crafting an example—a concise code snippet that demonstrates how the feature will be utilized. This approach may remind you of Test-Driven Development (TDD).

TDD serves not only as a means to test your software but also as an effective way to define your API and illustrate its usage. I discussed this concept in my article titled "Tests Are Good. But What Are Good Tests?"

By practicing TDD, you ensure low coupling between modules, which promotes modularity and maintainability. Moreover, it guarantees that your API will be easy to use. After all, who would want to write dozens of initialization lines before even getting to the actual testing? TDD allows you to focus on the essential aspects of your API, making it more intuitive and developer-friendly.

To create a user-friendly API, it's crucial to have it used by people who didn't work on its development. Your own bias, as the creator, can hinder accurate judgment of its usability. External feedback is invaluable for making necessary improvements and ensuring a more effective design. To do so, you can do API Review.

How to Organize an API Review ?

The best way to do API review is to write a code snippet that use your API, to present it to some developers and see if they understand what the code do.

It's essential not to reveal the methods' implementation during this review, as the focus is on the API design, not its implementation. It's better to do a first API review before writing the implementation anyway. Keep in mind this will be an iterative process. The API you imagine may not be implementable. So you'll have to iterate and do other API reviews as long as your API will converge.

Good and Bad practices

If you practice C++, Qt has a very good API Design guidelines. And below are some examples I've encountered in my career that can be applied to any language or framework:

An object should always be in a consistent state, and thus from its creation.

Do not defineinit(), create(), build(), etc... methods that must be called just after object creation:

var serviceClient = new ServiceClient();
serviceClient.initialize();
serviceClient.connect("http://myservice.com");

Prefer instead:

var serviceClient = new ServiceClient("http://myservice.com");

Or using a factory, so you can return null , or a Null Object Pattern if creation fail for any reasons:

var serviceClient = ServiceClient.create("http://myservice.com");

You API should be stateless.

Not only for REST API, but for any kind of API. Creating an object, or calling a method, should not depend on current state of application. In particular, it should not be needed to call unrelated method before using a function or creating an object:

LogManager.init();
ConfigManager.init()
Database.registerConfigManager(ConfigManager.create())
// I don't know what the line above do but if I don't call them, the line below crash.
var icone = new Icone();

If your object do need some other object to be initialized, request these objects as argument:

var configManager = ConfigManager.create();
var icone = Icone.create(configManager)

As a corollary, avoid using global variable or Singleton pattern, which makes your API stateful.

Another example are what I call the "Stateful getters", where you need to call some initialization method before you can call a getter:

object.computeSize();
object.getSize();

Solution is to use a lazy evaluation approach:

Object::getSize()
{
  if(m_size.empty())
    m_size = computeSize();
  return m_size;
}

If a method is present, it should be callable

Avoid throwing "Not Implemented" or "Not Supported" exception. It's very frustrating to find the exact function we look for using autocompletion just to figure out at runtime that we cannot use it.

Image readOnlyImage = Image.open(ïmage.png", "r");
readOnlyImage.write(someData); // throw NotSupported

If possible, prefer splitting in Interfaces:

ReadImage readOnlyImage = Image.openRead("image.png");
readOnlyImage.write(someData); // write is no part of ReadImage interface

Do not ask for parameters that can be deducted from others.

You just look like an idiot if you request for something you can deduct by yourself. It also avoids the need to handle inconsistent parameters:

Server.download(url="http://myserver.com/folder/image.png",
                protocol="http", 
                filePath="folder/image.png", 
                fileExtension=".png")

And good luck to handle the following case 😊:

Server.download(url="http://myserver.com/folder/image.png",
                protocol="ftp", 
                filePath="folder/sound.mp3", 
                fileExtension=".jpg")

Avoiding redundancy avoid you a lot of trouble:

Server.download(url="http://myserver.com/folder/image.png")

Makes the nominal use case trivial to do

90% of your user will do the same things. Makes this 90% doable in a single line of code. Don't provide a super generic API only to makes life easier for the remaining 10%.

var imgDims = image.getDimensions();
var widthDim = imgDims.find(dim) => dim.axis == WIDTH_AXIS);
var width = widthDim.range();

Even if an image can be a 3D multispectral time series video, 99% of peoples handle 2D images:

var width = image.getWidth();

Expose only what is strictly necessary.

Anything you expose can, and will, be used by user. And user will complain about anything that don't work. So don't expose that nice stuff you did during lunch or that internal function that you use to debug some specific cases. Exposing a feature is the PO's responsibility, not the developer one.

Avoid doing thing if the user hasn't explicitly asked you.

Some language enables you to call function at startup, when the library is loaded for example. Python has __init__.py, C# has[RuntimeInitializeOnLoadMethod()] tag, etc... Be very careful when using it. There is nothing more frustrating than experiencing crashes or error logs from a third-party when you haven't explicitly invoked any of its functions.

Also, keep this in mind:

Not because you did something complicated means you did something smart

Improve Your Unity Asset Usability With Samples

2023-06-01T17:10:00+02:00

No one read the documentation.

It is the sad truth. You have developed the most awesome Unity asset, using the most advanced state of the art ray-casted-signed-distance-field-backed-layer-based-mipmap-inspired-linear-fading technic, but no-one understands how to use it. So you wrote a documentation. But people won't read it. Because no one read the documentation anymore.

If you want people to use your work, you have to make it easy to use.

Unity Package Layout

Unity recommends a specific folder layout for your package. It looks like this:

If you respect this layout, you can import your package in Unity using Unity's package manager Add package from disk or Add package from git URL if you store your sources on a git repository.

Try to import the Unity's High Precision Framework package:

Click on Add package from git URL...
past the git url: https://github.com/Unity-Technologies/com.unity.gis.high-precision-framework.git
(for some reasons, you may have to add git+ if your git is stored on custom server:
git+https://my-company@dev.azure.com/MyProject/_git/my-unity-package)

The package is importing. This may take a few minutes. You will notice the " View documentation" that no one ever click.

But a much more interesting part is the "Samples". This list samples provided in the package.

If you click on "Import" button, Samples will be downloaded and added to your Project Browser.

How to create samples for your custom package

Samples are stored in Samples~ folder of Unity Package Layout:

But to be present as Samples in Package Manager, you have to reference them in package.json:

"samples": [
        {
            "displayName": "High Precision Sample",
            "description": "Contains sample scene for the high precision framework",
            "path": "Samples~/SampleScene"
        }
    ],

How to create good sample

Sample should contains a scene and should work out of the box when user load the scene.
Sample should be minimal, and focus on a single feature
Sample should be small. Ideally, a single scene with a single node and a single script.

Conclusion

People prefer copy/past something they don't understand than understanding something to make it work. That's why they won't read the doc. So instead of spending time on documentation, prefer spending time on writing simple, self-contained examples.

Also, keep this in mind:

Not because you did something complicated means you did something smart

How to locate DICOM in space

2022-12-20T10:04:00+01:00

I've worked in medical domain from time to time since 2010, especially to display DICOM images, and if I had a dollar every time I heard “This picture is upside down”, I think I could have bought a sandwich. Yes, that's not so frequent but when it happens, it quickly becomes a nightmare. So let's try to demystify DICOM image position.

Some Convention in 3D

I noticed that some convention that was obvious to me was not obvious to everyone, so let's remind them:

The vast majority of 3D software always use the same colors to draw axes:

When an axis is normal to the image, a circle with a dot indicates this axis goes toward you:

And a circle with a cross indicate this axis is moving away from you:

Coordinate system

A very important thing to understand is that coordinates of a point always refer to a reference system. In the example below, the coordinates of point P are in both case (2, 1), but depending on which coordinate system we use, its final position won't be the same:

When talking about Image Position (Patient) Attribute - (0020,0032) for example, one should understand in which space are expressed these coordinates. Same for Image Orientation (Patient) Attribute - (0020,0037). So, how to know in which space these coordinates are expressed ?

RCS - Reference Coordinates System

DICOM define two reference coordinates system depending on if the patient is a biped or a quadruped. It corresponds to the natural way of putting specimen in device (scanner, MRI, ...) so that patient space correspond to device space. This information can be retrieved with the Anatomical Orientation Type Attribute - (0010,2210). As states in the doc: “If this Attribute is not present, the default human standard anatomical position is used”, that is to say Biped.

BIPED

X: from right to left
Y: from front to back
Z: from foot to head

QUADRUPED

X: from left to right
Y: from dorsal to ventral
Z: from tail to head

Image space

DICOM also define an image space with origin in center of first pixel. Note that most render engine consider origin in corner of texture, which cause a lot of “half pixel shift” issues in medical applications.

X: along rows, unit of 1mm
Y: along column, unit of 1mm
Z: cross product of X and Y to form a unit direct coordinate system.
Origin: Center of first pixel of image

When talking about 1st pixel of image, I'm not referring to top left one or bottom or whatever. I'm referring about the first value in pixel buffer. For example, with a 256*256 image:

If you do a simple DICOM image viewer, you can just locate 1st pixel in top left and that's it. But if you do a 3D viewer and have to deal with patient orientation, oblique slices and so on, thing will becomes much more complicated, and you should better not focussing on displaying 1st pixel in top left.

Locating image in patient space

Now serious things begin. Let's remind the 3 tags involved here:

Image Position (Patient) Attribute - (0020,0032) gives Image space origin in patient space.
Image Orientation (Patient) Attribute - (0020, 0037) gives Image space x and y axis direction in patient space. z can be retrieved with cross product between x and y.
Pixel Spacing Attribute - (0028, 0030) gives length of Image space x and y axes

We can then build a transformation matrix from image space to patient space:

Applying this transformation matrix to image properly locate it relative to patient:

Before taking care of image orientation on screen, you first have to ensure the image is properly located into patient space. Else, you will just do shotgun debugging by blindly flipping and rotating the image until that particular one is well oriented. And you will have a bug for the next one.

A few words about Anatomical planes

Before talking about image orientation on screen, we should introduce Anatomical Planes. Medical domains defines 3 cutting planes, relative to each of 3 patient main axes:

Sagittal: Normal to X axis
Coronal: Normal to Y axis
Axial (Transverse): Normal to Z axis

Defining if an image is along Sagittal, Coronal or Axial plane is easy in case of axis aligned images, but in case of oblique slicing, this may be more difficult. To do so, we have to find which of the Sagittal, Coronal or Axial plane is the closer to slice plane.

Let's first compute image normal, expressed in patient space. It is the cross product of image's X and Y axes, returned by Image Orientation (Patient) Attribute - (0020, 0037).

Now, let's find which of the Sagittal, Coronal or Axial axis is closest to slice's axis. This can be done by computing dot product between slice normal and X, Y, Z patient space axes. But as we have slice normal directly expressed in patient space, dot product between X, Y, Z patient axes are directly the coordinates of slice's normal:

If the highest value is Zimg.x, it means the slice is closest to Sagittal plane, and thus can be considered as a sagittal slice.

Displaying a slice on screen

Until now, I didn't talk about slice orientation on screen. I never talked about Up or Down. This is because these concept is a matter of Camera, not position in space.

Let's assume you have a Coronal slice of the patient's head. How would you want to see it on screen ? Like on the left or like on the right ?

Note that both orientations are valid. In both cases, Z axis goes from foot to head and X axis goes from patient's left to right.

To decide which is the better, a convention defines how Sagittal, Coronal and Axial images should be displayed on screen. I didn't find any literature affirming this convention, but it seems the most widely used among medical software.

Note that the convention is not the same for BIPED and for QUADRUPED.

Sagittal Biped

Patient Z axis along UP screen vector
Patient Y axis along RIGHT screen vector

The patient is standing up and look to the left of the screen.

Sagittal Quadruped

Patient -Y axis along UP screen vector
Patient -Z axis along RIGHT screen vector

The patient is lying down is standing up and look to the left of the screen.

Coronal (Biped and Quadruped)

Patient Z axis along UP screen vector
Patient X axis along RIGHT screen vector

The patient is standing up and face you. Its left side is on the right of the screen.

Axial (Biped and Quadruped)

Patient -Y axis along UP screen vector
Patient X axis along RIGHT screen vector

Patient is lying down and look to the top of the screen.

Let's write some code

Let's wrap this into some pseudo-code:

vec3 xImageInPatientSpace, yImageInPatientSpace = readtag("(0020,0037)");

vec3 sliceNormalInPatientSpace = xImageInPatientSpace.cross(yImageInPatientSpace).normalize();

Orientation orientation;
float maxCoord = max(abs(sliceNormalInPatientSpace.x),
                     abs(sliceNormalInPatientSpace.y),
                     abs(sliceNormalInPatientSpace.z));

if (maxCoord == abs(SliceNormalInPatientSpace.x))
    orientation = Sagittal;
if (maxCoord == abs(SliceNormalInPatientSpace.y))
    orientation = Coronal;
if (maxCoord == abs(SliceNormalInPatientSpace.z))
    orientation = Transverse;


if (orientation == Coronal && biped) {
  right = yPatientInWorldSpace;
  up = zPatientInWorldSpace;
} else if (orientation == Coronal && quadruped) {
  right = -zPatientInWorldSpace;
  up = -yPatientInWorldSpace;
} else if (orientation == Sagittal) {
  right = xPatientInWorldSpace;
  up = zPatientInWorldSpace;
} else if (orientation == Transverse) {
  right = xPatientInWorldSpace;
  up = -yPatientInWorldSpace;
}
camera->lookAt(target, right, up)

This will really depend on your application, which lib you use to read DICOM, which render engine you use for your display, but the main idea is here. From Image Orientation, you can compute an UP and a RIGHT vector, and then ask your camera to look at some point using the given UP and RIGHT orientation. Most render engines have this kind of feature.

Not axis-aligned image

The code above ensure having patient space always aligned with screen axes. But what if you have an image which is not aligned with patient axes, like the case below ?

We have a perfectly Sagittal acquisition. The slice normal is along X axis, but image rows are not aligned with patient Y axis. In this case, what do you want ?

Left: Patient aligned with screen. Image not aligned with screen.
Right: Image aligned with screen. Patient not aligned with screen.

Well... it depends on what you want, but you have to choose. You cannot have both image and patient aligned on screen.

The code presented previously will give you the Left result. Patient axes aligned on screen. If you want the Right one, you should add an extra step to ensure left and up vector are aligned with image X and Y vectors:

vec3 xImageInPatientSpace, yImageInPatientSpace = readtag("(0020,0037)");

vec3 sliceNormalInPatientSpace = xImageInPatientSpace.cross(yImageInPatientSpace).normalize();

Orientation orientation;
float maxCoord = max(abs(sliceNormalInPatientSpace.x),
                     abs(sliceNormalInPatientSpace.y),
                     abs(sliceNormalInPatientSpace.z));

if (maxCoord == abs(SliceNormalInPatientSpace.x))
    orientation = Sagittal;
if (maxCoord == abs(SliceNormalInPatientSpace.y))
    orientation = Coronal;
if (maxCoord == abs(SliceNormalInPatientSpace.z))
    orientation = Transverse;


if (orientation == Coronal && biped) {
  right = yPatientInWorldSpace;
  up = zPatientInWorldSpace;
} else if (orientation == Coronal && quadruped) {
  right = -zPatientInWorldSpace;
  up = -yPatientInWorldSpace;
} else if (orientation == Sagittal) {
  right = xPatientInWorldSpace;
  up = zPatientInWorldSpace;
} else if (orientation == Transverse) {
  right = xPatientInWorldSpace;
  up = -yPatientInWorldSpace;
}

// Extra step to handle Image screen aligned {
right = findClosestVector(right, [xImageInPatientSpace, yImageInPatientSpace])
up = findClosestVector(up, [xImageInPatientSpace, yImageInPatientSpace])
// }

camera->lookAt(target, right, up)

With findClosestVector:

// Find among vectors, the vector which is the closest to refVector.
// All vectors must be normalized.
vec3 findClosest(vec3 refVector, vec3[] vectors) {
  float closestDot = 0;
  vec3 closestVector = vec3(0);
  for(v in vectors)
  {
    float dot = abs(refVector.dot(v));
    if(dot > closestDot)
      closestVector = v;
  }

  if(closestVector.dot(refVector) > 0)
    return closestVector;
  else
    return -closestVector;
}

Conclusion

I think That video perfectly summarize the kind of argument you may have when your user require “having images always aligned on screen and patient looking to the left”.

https://www.youtube.com/watch?v=BKorP55Aqvg

The Retrobinet

2022-11-16T16:24:00+01:00

Do you have difficulties choosing between creating new features and fixing bugs ? Maybe you should give a try to the Retrobinet. A sprint retrospective format to trigger discussion about priorities between maintenance and feature.

The concept

Retrobinet comes from "Robinet", the French word for Faucet. The idea is to have two faucets: one for the bugs and one for the features, that fill up a bucket representing your team velocity. You have a 10 litter bucket. You cannot fill up more than 10 litters.

Question: How do you adjust the faucets ?

To trigger discussion and ideas, you can imagine what do you do if you have:

5L feature, 5L bug ? (this one is easy 😉)
10L feature, 5L bug ?
5L feature, 10L bug ?
15L feature, 15L bug ?

You cannot put 15L in a bucket of 10L. So a strategy should be defined. Bug first ? Feature first ? 50/50 ? 80/20 ?

Let your team think about this during 10 minutes. The first thing they may argue is:

— Things are not that simple in real life !

Yes, so what information is missing ? Is this information present in the ticket tracker ? Does having such information will really help you ? If things were that simple, what would you do ?

Things are not that simple in real life, but this exercise highlight that prioritizing imply sacrifices. If you don't want to sacrifice anything, you can just let the bucket overflow, and you'll get what you get.

An introduction to Machine Learning applied to Real Estate Estimation

2022-09-19T11:33:00+02:00

I'm currently looking for a house in Montpellier, and I'm still wondering if the price displayed could be negotiated or not. So instead of thinking for myself (I'm very bad at business…), I prefer to let the computer do it for me.

To have a rough estimation of a property, you can get the average price per square meter and multiply by the property surface. But you know that lots of criteria can influence the final price. For example, the land area in case of houses. Let's see how machine learning can help us to refine our estimation by taking in account more than only living area.

Building our Model

We will suppose, for this very simple model, that real estate prices mainly depend on living area and land area. Our goal is to have a program that takes those 2 values as input and give us the price of the house:

house_price = compute_price([living_area, land_area])

By convention, inputs of the model are called the “Xs", and output are called the "Ys". The model is a function that compute a Y from a given X.

Unfortunately, there is no such thing as “The Model to rule them all”. There are dozens and dozens of models, more or less complicated, more or less easy to use, more or less adapted for certain category of problems. For example, assuming a “Price per square meter” is already a model in itself. You assume price grows linearly with living area. It may not be the case, it may grow exponentially, or stagnate at some point, or do some crazy things like “A house less than 100m² cost €300.000 and a house more than 100m² cost €500.000, and that's it”. But you assume using a linear variation is a good approximation of what happen.

We can assume final cost will also vary linearly with land area. The bigger the land, the more expensive the house. So let's create a simple model that assume house price is a linear function of living_area and land_area.

In Python, we will use the linear_model module of sklearn package:

 from sklearn import linear_model

Training our Model

To train this model, we need data. There are several ways to get real estate transaction data. You can get them for example from https://www.immo-data.fr/. I won't detail how to get data here, but let's assume you manage to have a few dozens of real estate transaction over the last year on a given town. You know, for each house, its living_area, its land_area, and its price. For each couple [living_area, land_area], you have a corresponding price. This gives us 2 lists:

The list of [living_area, land_area], which are inputs of the model, the “Xs”.
The list of price, which are output of the model, the “Ys”.

# [living_area, land_area] X = [[137, 800], [120, 400], [125, 425], [130, 1500], [250, 1825], [351, 1612], [203, 4135], [173, 1690], [115, 622], [91, 388], [160, 2854], [106, 913], [86, 471], [80, 767], [135, 1497], [217, 2000], [167, 1796], [98, 3544], [120, 1007], [101, 3286], [134, 1108], [180, 1854], [137, 1784], [117, 1570], [176, 1743], ]

House price

Y = [629000, 599000, 594000, 650000, 840000, 1074648, 590000, 625000, 700540, 465170, 1000000, 430000, 503900, 370000, 632000, 1200000, 700000, 655750, 564000, 600000, 568400, 815000, 650000, 485800, 724700, ]

We train our model using those data. It will learn how price vary in function of living/land area in this particular town.

 model = linear_model.LinearRegression().fit(X, Y)

Using our Model

When a new house for sale appears in that city, you can run your model to see if the price displayed is consistent with the local market.

 living_area = 110
land_area = 880
house_price = model.predict([[living_area, land_area]])

In this case, the model gives a price of €589.010. If this house is for sale for €590.000, that seems fair. If it's for sale for €650.000, you probably have room for negotiation.

Conclusion

Machine learning is a huge topic that has existed long before the trend of deep learning, and it exists model that are much simpler and require much smaller training set than deep neural network.

House pricing depend on various things like condition of the house, its location, the year of construction, its energy balance, etc… It could be a good exercise to build a more complex model taking in account those criteria. 😉

I let you the full python code. To run it, copy/past into a run.py file and type:

python run.py

You may have to install sklearn with:

pip install sklearn

 from sklearn import linear_model

X = [[137, 800], 
     [120, 400],
     [125, 425],
     [130, 1500],
     [250, 1825],
     [351, 1612],
     [203, 4135],
     [173, 1690],
     [115, 622],
     [91, 388],
     [160, 2854],
     [106, 913],
     [86, 471],
     [80, 767],
     [135, 1497],
     [217, 2000],
     [167, 1796],
     [98, 3544],
     [120, 1007],
     [101, 3286],
     [134, 1108],
     [180, 1854],
     [137, 1784],
     [117, 1570],
     [176, 1743],
]

Y = [629000, 
     599000, 
     594000, 
     650000, 
     840000,
     1074648,
     590000,
     625000,
     700540,
     465170,
     1000000,
     430000,
     503900,
     370000,
     632000,
     1200000,
     700000,
     655750,
     564000,
     600000,
     568400,
     815000,
     650000,
     485800,
     724700,
]

model = linear_model.LinearRegression().fit(X, Y)

living_area = 110
land_area = 880
house_price = model.predict([[living_area, land_area]])

print("Estimated Price: ", house_price)

Ps: I would like to thank the Odysee team that introduced me to machine learning domain. "It was short but intense" 😉

Scrum Retrospective: Archetype

2022-09-08T10:42:00+02:00

A few years ago, a hiring slot was opened in my team and our manager asked us which profile will be the most useful. We were a dream team, the best of the best. We lacked no skills ! And we didn't really know how to orient the job description. Should we hire a specialist ? A swiss knife ? A facilitator ? Hard to tell. So I've created a retrospective focused on team and individual profile which I had called:

"The Archetype retrospective"

Description

This retrospective is based on Role play and character creation. You have to chose between various archetype: The soldier, the Mage, the Paladin, etc... each with their strengths and weaknesses. So I defined a list of archetypes and asked to each team member to assign an archetype to each of its colleague, including himself. I end up with the following personas:

Benefits

This retrospective has 2 benefits:

First, this allows everyone to realize the image they give off. Sometime, there is a consensus on a person. Sometime, a person is seen differently by different members. It helps people to better know themselves.

Second, once you agreed on each member's archetype, you can plot the global team profile by accumulating each archetype profile. Goal is not to have a perfectly balanced team, this is an indicator to help you analyzing your team and should be interpreted. In my case, I end up with a profile like this one:

At first, one could think our team were lacking of team spirit and organization and would greatly benefit of hiring a "Paladin" profile. But our team worked well. There was a very good atmosphere and our release were always on time. But we noticed that we were lacking of polyvalence. And that was true. We were skilled, motivated and efficient developers, but no one was really "transverse". We came to the conclusion that, instead of adding another specialist, a DevOps polyvalent profile would be much more valuable for us.

Conclusion

This retrospective requires people to open themselves and accept judgement, which can be difficult for some. That's why before doing it, you should be sure you know your team, and it will accept this exercise.

Also, presenting the accumulated graph to management can be dangerous. They may be tempted to always have a balanced team which is not the goal.

I leave you the PowerPoint below. Let me know if you have tried this retrospective in your team and how it went 🙂

Archetype_retrospective.pptx Télécharger

An Introduction to Electron Microscopy

2022-06-27T09:30:00+02:00

In this article, I'm going to talk about something that has nothing to do with software development, but something I enjoyed working on and would like to put down on paper before I forgot everything.

Indeed, I will talk about Electron Microscopy, a topic I worked on when I was at Thermo Fisher company. All the information here are taken from the excellent playlist of Professor Grant Jensen: Getting Started in Cryo-EM with Professor Grant Jensen that I strongly encourage you to watch if you work in electron microscopy field, or if you have a strong appetite for knowledge that is useless in everyday life.

What is an Electron Microscope ?

This is an electron microscope:

It is a big box that measure about 3.5 meter high, connected to an army of computing cluster, a big power plug, some water and air tubes and sometimes a liquid Nitrogen tank. And it can do SCIENCE ! You can literally count the atom composing a transistor or see a COVID-19 virus entering a cell with this machine. For example, below is a spike of coronavirus reconstructed from CryoEM acquisition. This is basically what is inside the vaccine and that teach your cells how to recognize the virus:

So, is it just a microscope with electronic parts ?

Yes, there are electronics in Electron microscope. But it is called Electron Microscope because, instead of using Photon like classical microscope you use in biology course, it uses Electrons. To understand why Electron are better than Photons, let's have a look at how a classical microscope works:

Looking to an object is measuring how photons interact with this object. You light up the object from below and use a lens to scale up the resulting image. This works as long as photons are smaller than this object, because the size of a pixel of the resulting image cannot be smaller than the size of the photo that create this pixel.

The problem with photons is that they are much bigger than atoms. An Atom is ~1 Å, which is about 0.1 nm. A photon is ~4000 Å, so 4000 time bigger. On the other hand, an electron is ~0.01 Å, a hundred time smaller. And if you think about it, an atom is just a nucleus with electrons spinning around. So yes, an electron IS much smaller than an atom.

In the end we just have to build an electron flashlight to illuminate our object, an electron magnifying glass to magnify the image and an electron camera to take a picture, and we have an electron microscope 😎

Parts of an Electron Microscope

If we open our electron microscope, we have the Column, which is the heart of the system:

This column look like an optical microscope, except that it is upside down. The light is at the top, then you have a few lenses to diverge/converge the light, then the sample, another bunch of lenses and finally the sensor at the bottom.

Electron Gun

2 parts compose the Electron gun :

The cathode, which generates the electron.
The accelerator, which accelerate the electron to the good direction.

Cathode is a very simple object. It is basically a tungsten wire bent into a V shape:

We then apply a very high tension to this wire, and the spiky shape will naturally emit electrons thanks to the Thermionic effect and the "Effet de pointe" (sorry, didn't find English references).

So we apply a negative tension, typically -300 kV. As an electron has a negative charge, it will be pushed back. On the other end, the ground of the Electron Microscope is kept at 0V potential, so electron will be attracted by the bottom of the microscope. Furthermore, to properly accelerate electron, they go through an accelerator which maintain a gradient of potential that smoothly go from -300kV to 0V:

By the way, an electron emitted with a 300Kv tension has an energy of 300 kilo electron volts, or 300keV. And current microscopes are able to generate electron one by one.

Lens

An optical lens converges and diverge light rays. It is the same for electron: it converges and diverge electrons "rays" except that electron lens are not made in glasses or transparent material. They are made with magnetic field generate by electromagnet:

These magnets heat when in use. And heating a copper wire change its resistivity, which change its magnetic field, which is not a good thing for us because it's like changing the focus while taking a picture with a camera. So to avoid this, we have to cool down all these elements with water. Remember there is a 300,000 Volt electrical device operating a few inches away…

But how does an electron lens work ? If we compare to optical lens, you probably have seen in college that rays emitted from a point converge on a single point to form an image, with some stories of focal point, focal distance, etc… :

Actually, there is a reason why rays converge in this particular points. To understand why, we have to remember that light is a wave, so the diagram above should look like:

The interesting thing about waves is that they can add themselves if they have the same phase:

or subtract themselves if their phases are opposite:

It is the same for lights. Traversing a transparent material change the light frequency, and traversing different thickness change the light wave phase when it comes out of the lens. Also, when a wave comes out of a surface, it is emitted into every direction. So we can consider that every single point of the universe potentially have an image of the initial object, but for most of these points, waves arrive with the wrong phase, and they destroyed each others.

But there is a single point where all the light rays arrive with the same phase, and all contribute together to form the image:

It is the same for electron. Electrons are also waves, and when they pass through a magnetic field, we can change their phase like a lens do for light.

Sample Holder

One thing to know about Electron microscope column is that they are maintained in a nearly perfect vacuum. Any "air" molecule can interact with the electron beam and decrease acquisition quality. In that case, how do you place the sample inside the acquisition chamber ? With an airlock of course 😉

On the side of the microscope, there is a tube with a valve where we can insert the sample holder:

The sample holder also contains a liquid nitrogen tank to keep the sample frozen during the whole acquisition.

Energy filter

EM Acquisition energy is measure in kilo-electron volt (keV). A typical CryoEM acquisition is done at 300keV. For various reason, it is very important that electrons that reach the detector really are at 300keV. We don't really care if electron traversing the sample are not at the good energy, but electrons reaching the detector must be at 300keV.

To enforce this, there is a system placed before the detector called Energy filter, which goal is to filter electrons that are not perfectly at 300keV.

To do so, the energy filter will generate (yet another) magnetic field to bend the electron beam up to a diaphragm. Electrons with an energy of 300keV will pass through a hole in this diaphragm, whereas other electron will not follow the same trajectory and will crash into the shield:

Detector

Today's detectors are able to detect electron one by one, and a raw electron microscope acquisition is far far away from beautiful colored images you can see on scientific papers. Indeed, a raw image looks like this:

Each white dot is an electron impact. But by accumulating lots and lots of images, we can obtain this:

And finally, with lots of mathematics and processing, we can obtain this kind of images that, in addition to being beautiful, really helps scientist and biologist to understand the deep behavior of nature:

How to Work Efficiently With Git ?

2022-06-07T09:31:00+02:00

Git is a tool, and like any tool, there is a big difference between using it and using it properly. I will describe in this article the workflow I'm using in my day-to-day developer's job.

Starting to Work on a Ticket

When I start working on a Bug, on a User Story, on a Task or on whatever, I always start by checkout a branch containing my name, a short description of the issue and ticket number. It looks like :

git checkout -b tan/fix-event-crash-MIC-1831 origin/master

Saving my work

After a few minutes or a few hours, depending on the complexity of the task, I end up to commit my modifications. I rarely use command line for commit, but use git gui instead:

git gui

And I systematically push my branch, also with git gui push button. This has 2 advantages:

If I screw up my repos, I can still reset on the last time I pushed by branch.
If my computer crash for any reason, I have a backup of my work.

I also use gitk a lot to see current status of my branch:

gitk

Here I can see that I'm based on master, thanks to the remote/origin/master label. I have 3 commits in my branch and my branch is currently pushed, thanks to the remotes/origin/tan/feat2 label being the same as local tan/feat2. So my work is saved.

I really commit/push A LOT. I push several times a day, and sometime up to several times per minutes for very short modifications.

Retrieving Others' Work

I regularly rebase my branch on current master to retrieve other's work and address conflicts as soon as possible.

git fetch
git rebase origin/master

Note that I rebase, and not merge master into my branch. I want my history to be linear, and the only merge points that appear in history are merged of feature branches into the master branch.

Always Push at the End of the Day

Whatever the status of my code, I always commit my work at the end of the day and push it. I never know what can happen. If I'm in a draft state, I just put everything in a commit and prefix commit message with “WIP”:

Modifying the Yesterday's WIP Commit

After a good night, I wake up and wonder what the hell I was doing yesterday. I clean up my mess and want to do a clean commit. But I commit an ugly “WIP” commit yesterday. Here comes the awesome “Amend Last Commit” git gui checkbox:

This checkbox let you modify the last commit, rewrite commit message, stage/unstage files etc… So I really have no reasons to not commit a WIP work.

Interactive Rebase

I often do intermediates WIP commit, to explore a path until it leads to something concrete. I also frequently do small dumb fixes or typo correction that should go on older commit. To avoid having a lot of intermediate commits that goes back and forth, I practice interactive rebase. Interactive rebase is a wide topic. You can have a look here to have more details.

In the example below, while fixing a crash, I fix a typo in documentation I previously wrote. To indicate it, I prefix the commit message with an f for fixup. I would like this commit to be squashed with the 2nd add doc commit, and then I would like to squash the WIPcommits with the fix feat 1 commit.

To do so, I do an interactive rebase:

git rebase -i origin/master

And go from this:

to this:

Now my branch is clean, I can publish the merge request and ask for review.

Conclusion

This workflow is well adapted to Trunk based development, but can also be used for any git branching models. The most important points are:

Rebase branch on master instead of merging master in branch.
Always push branches, even draft and work in progress branches.
Learn interactive rebase.

(Bonus for Git masters) Retrieving Temporary Work of Others

Ideally, development branches should not live more than a few days, or even a few hours, so you should never have to use the work of other developers before it has been integrated in master. But in complex monolithic legacy projects, integration may take days or weeks… In such a case, your work may depend on someone else work that is still not integrated. There are many ways to do this: Rebasing branch on each other, cherry-picking commits, merging branches, … I will describe the method I use.

Assuming I am in this situation. I already have some commit, starting from master:

But I some bob/fix-db branch to continue my work. This branch is in the integration pipeline, but won't be available before next week. In this case, what I do is:

# ensure everything is up to date
git fetch

# save my work. 
# If I screw up, I can still reset to what I just pushed
git push origin HEAD:tan/feat2

# get an overview of my work, and keep it on the side
gitk&

# now serous things begins. Reset my branch to common trunk
git reset --hard origin/master

# I start by merging the branch I need
# --no-ff force git to create an explicite merge commit
# and not flatten the history in case of fast forward.
git merge origin/bob/fix-db --no-ff
# Maybe I need several branches to continue my work. 
# I do this for each branches.
git merge origin/ben/fix-build --no-ff

# now, I retrieve my commits from the opened gitk
git cherry-pick ...
git cherry-pick ...
git cherry-pick ...
# using range git cherry-pick sha1...sha2

# And finally, check my branch looks good
gitk

History should look like this:

I can see that:

My branch tan/feat2 start from master
It first merges the bob/fix-db. I see this thanks to commit Merge branch 'bob/fix-db' into tan/feat2.
The label remote/origin/bob/fix-db is just before merge point, so I'm up-to-date with Bob's work.
I've added 4 commits on top of Bob's work.

The main advantage of merging dependencies and cherry-picking your own commits, is that you and your colleagues can clearly see in history what is your own work and which branches your work depend on.

There is also another big advantage. Imagine the branch bob/fix-db broke something during integration and Bob had to fix its branch. By cherry-pick its commits or by basing your branch on its branch, you cannot see this. With the merge technique, you can directly see it:

git fetch
gitk

You notice that the remote/origin/bob/fix-db has disappeared, indicating bob did something to its branch. In this case, just repeat the procedure described previously.

This procedure can also handle case when pbr/fix-db has finally been integrated in master. In this case, the git merge origin/pbr/fix-db will tell you this branch is already merge and there are nothing to merge.

Ideally, you should repeat this procedure until all dependent branches has been merged, and send your merge request only when you have only your modifications in your branch.

As said at the beginning, this technique is really a workaround to handle very long integration pipeline. If your work depend on other's one, it is always better to wait their work to be integrated than doing high-flying gitology.

How to Organize a Backlog ?

2022-05-20T09:39:00+02:00

Product backlog is really the barebones of agile scrum teams. It help Product Owner to communicates vision to stakeholders, it help dev team to anticipate future needs and adjust design accordingly and it help Scrum master analyzing how team can be more efficient in its work.

Unfortunately, backlog is often a mess filled with "could be useful" ticket , "will never be fixed" bugs and other "I got this idea the other day when taking a shower" stories. We will see in this article some tips to help maintaining your backlog clean and focusing on what really matter.

Addressing stories

I won't detail the different level of Program/Initiative/Epic/Story here. I've always been closer to production team than marketing team, so I have not so much experience regarding Epics or Program ticket level. Lets just assume a story represent something needed by a user that can be implemented in less than a few weeks of effort by dev team. As we are in an agile environment, user require this feature now and will certainly change its mind in a few month. So this story should be addressed quickly to be valuable for our user, and as a consequence, for us. This lead us to the first statement:

A story is not a long term vision.

Story should be addressed in the next few months, in current or next quarter. A story planed to be addressed in a year has lot of chances to be obsolete and no more adapted to user need, and just pollute backlog during months with useless information. That's why Product Backlog should only contains the detailed stories for the next 2 quarters at worst.

— Yeah, but I have lot of important ideas !

That's cool ! But, either your ideas are priority for next release, so they are in backlog, either it's just some good ideas but not required for next releases, in this case they should not appear in backlog to avoid drowning important stories.

— Yeah, but I want to keep trace of it !

Yes, I know it's very critical to keep trace of those 2 minutes cogitation you had during your breakfast... You then have 2 solutions:

Solution 1

you really want your team to spend time on it. You want to spend money refining this idea, having a rough estimation of its efforts and its ROI but you are still not ready to plan it in a release. You can store it in something called a "Funnel". It can be a completely separate backlog like Productboard, Aha, a dedicated Jira board or even lighter tools like Miro or Trello. It can also be a filter in main backlog to hide these story by default. Once you find this story valuable enough, you can move this ticket to product backlog and really refine it in order to implement it in short term.

Solution 2

Your idea is just an idea. You can talk about it during coffee time, takes inputs, etc... but keep it in your own PO's TODO list until you consider it important enough to be placed in Funnel.

Remember, the more stories are in backlog, the harder it is to focus on what matter.

— And what about stories that were top priority at some time but were canceled because of reasons ?

As you said. They were canceled. So stop lying to yourself. Either you will do it, either you won't. Either you implement it for next release, either you remove it from backlog to focus on what matter. You can put it into Funnel or completely cancel it. Cancelling a story in Jira or any other board does not delete the ticket, it is still somewhere and can still be reopened in the future. (Spoil: this will never ever happen...).

Addressing bugs

Stories are ticket that should be planed in short/middle term depending on their ROI. Bugs should be handle differently, more like a continuous flow. You can stop the flow of story. You decide to open or close the faucet. You can close the faucet and stop any activity if you want. Unfortunately, you cannot do this for bugs. Bug flow is controlled by user. And if you are falling behind to fix bugs, they will pile up and overwhelm you.

As a rule of thumb, put bugs in top of backlog so that they will be addressed in next sprint. Do not be tempted to accumulate bugs to do a "stabilization sprint" just before release. By "address", I don't mean "Fix". To address means giving a solution to user. You can fix it, you can find an acceptable workaround, or you can purely cancel it considering bug is too costly to address. You would be surprised by the amount of user who can accept a bug if you explain them gently why it cannot be fixed. And you would be even more surprised by amount of user that would find a workaround as valuable as the actual fix. Finally, handling bugs in backlog can be summarize by:

- Bugs are top priority.
- Either you fix it, either you close it.

That's theory. In practice, I encountered a case where it was a better idea to accumulate bugs and fixing them by bunch. It was an old C++/COM legacy software. Deploying a dev environment required a day or two, copying file here and there, setting environment variable, downgrading to an old Visual Studio, etc... There was no activity on this product anymore but we still receive some bugs from time to time. In this case, we agreed to spent a "bug fix sprint" once or twice a year to fix all bugs at once and release a new patch.

What about long term vision ?

We just saw that product backlog should gives a few months, maybe a years of visibility, no more. But How do you share your long term vision ?

You don't share a vision with tickets. You share a vision with discussions and with coherence on what you do day after day. If you put in backlog stories that don't have any link to each other, no one will understand where you want to go. If you put stories that Tell a story, a common thread will emerge and people will understand your vision.

What about the past ?

Backlog is NOT an overview of all the thing that was asked one day by some customer. It is NOT an overview of all the missing and broken stuff in your product. It is an overview of what you are going to do in the next months. So stop archiving every single request in the backlog.

In the end, what does a clean backlog look like ?

A good theoretical backlog look like this :

Backlog is alive. Priorities can be changed and stories can be cancelled at any time. But the idea is to keep good visibility in short terms by having well defined stories on top and have a rough idea of what is going middle term with stories to be refined at the bottom.

Remember: The Backlog is not a tool to show everyone all the great idea you have for your product. It is not an archive of everything your users have ever asked for. It is a tool to focus on what is important. The hardest part of a PO's job is not overfilling backlog. It is on the contrary to keeping it short and valuable.

How to Write a User Story ?

2022-05-05T11:24:00+02:00

Ah, stories... Dev team complains they are not enough specified, Product Owner complains he doesn't have time to write them and Scrum Master complains they are a meaningless mess. But writing a story should not takes than much time and it doesn't have to be detailed to be clear. Let's see how to be efficient in writing stories and see some good and bad examples.

A story tells a story

First, goal of a stories is to synthesize a user's need. It should create empathy so that developers understand how they can help user. So a story should tells a story. It should contains a protagonist who aspires to a goal. And to reach this goal, he needs something from you, he require your help. You are the support. You are the Sam Gamgie that will help Frodo throwing that damn ring into Mount Doom. So a good story should contains a protagonist, a need and a goal. And a very good way to always have these 3 points is using the famous:

As a [protagonist], I need [something] in order to [achieve a goal].

I really hated this phrase at the beginning of my career because I didn't understood the important part was not its structure "As a ... I need ... in order to ...", but the 3 points we just discussed before. Lets analyze them more in details:

The protagonist

A lot of stories start with "As a user". That's the worst way of starting a story because it gives no information. Anyone using your code is a user, including yourself. Imagine starting a tales by "Once upon a time, in a place, lived a character."... That's the case for any single story told since the beginning of mankind, so try to be more specific.

In our stories, protagonist can be:

An identified customer:
- As Renault Truck, ...
- As Walmart, ...
- As GE Aviation, ...

A persona:
- As portal administrator, ...
- As front-end developer, ...
- As game artist, ...

Or even an identified team if you are developing internal tools:
- As Kernel team, ...
- As Marketing team, ...
- As ML team, ...

The need

The need should be expressed from protagonist point of view, and not from developper's point of view. Don't write "update library version" but "be compatible with new hardware". Don't write "Integrate elasticsearch" but "have search bar in main page". The technical solution will be decided by developers once they will have understood the need.

The goal

The goal gives context to dev team and let them understand the scope and criticize the need. Maybe you won't be able to achieve your goal with just what you're asking for ? Maybe you don't need that at all to achieve your goal and there is a much simpler solution ?

Yes, I know, Gandalf could not asked this to the Eagles because of reasons, bla bla bla, but that's not the subject. The subject here is that, if dev team know why they implement such and such feature, they won't lose energy over-designing, they will be more inclined to propose a suitable solution, and maybe they will tell you "you can already do that".

A story should be demonstrable

Last but not least, a Story should be demonstrable. As a developer, how can you prove you implement something ? As a QA, how do you validate new behavior is the expected one ? As a PO, how do you prove you add a new feature in your product ? As a team, how do you show the new feature during sprint review and get feedback from stakeholders ?

There are several ways to define acceptance criteria. You can write detailed specification that no one will read anyway, or you can use simpler format like "Given ... When ... Then ..." or my favorite one: How to Demo. let's see a small example.

As an administrator, I want to see the list of currently connected users to see who will be impacted by maintenance.

*How to Demo*

- Create a few fake user
- Go to Admin Page.
- Click on "Current Users" button
=> List of fakes users should be displayed

This small scenario gives us a LOT of information and questions:

You need to populate database with fake data, or at least a dataset. Do you have one ? Are you able to create those fake data ?
Go to Admin page. Does it already exist ?
Click on "Current user" button. Do we have mockup for this button ?
List of fakes users should be displayed. Do we have mockup for this list ?

So you can iterate on the story until it is ready, and avoid dev team working on it to discover in the middle of the sprint they lack half of the information.

This How to demo also tells to developers when to stop. If you don't explain how to demo, dev team may be tempted to extrapolate and work on the maintenance part. Here, development ends when we have the list of connected users, we don't care about maintenance operation, this is another topic.

Having an How to demo also tells to QA how it can validate the story. And finally, It gives the scenario to present to sprint review.

Example

To conclude, here are some bad and good examples.

Implement GPU picking

Acceptance criteria:
Picking should be done on GPU

As Airbus CAD team, I want picking on object to be more responsive in order to improve user experience

How to Demo:
Given attached data displayed in viewer
When I select objects
Then Selecting an object should not takes more than 500ms.

Example

As a user, I need to update CUDA to 11.4 in order to support new Ampere GPU.

How to Demo:
- Open properties in about dialog and show CUDA version
=> version should be 11.4

As IT department, I need application to support new Ampere GPU in order to move to new infrastructure.

Note: Ampere GPU require CUDA >=11.4

How to Demo:
- Deploy application on new infrastructure
- Load some data
=> Application should works correctly

- Execute test suite on new infrastructure
=> Test should be green

Example

As a User, I want elasticsearch in order to have a search bar

How to demo:
- Load main page
=> There should be a search bar (see attached mockup)

As a buyer persona, I want to have a search bar in main page in order to search product by name

How to demo:
- Load main page
- search for product in search bar (see attached mockup)
=> corresponding products should display

Tests Are Good. But What Are Good Tests ?

2022-04-25T09:02:00+02:00

Tests are an essential part of development. Well tested application have more line of code in tests than in actual code. But not all tests are equals and, unfortunately, some tests are written only for the sake of reaching a code coverage target or making QA happy to see test base increasing.

What is a test anyway ?

Let's kill a myth. Autotests had never ensured your application works correctly. It ensures your application works like before. It is up to developer, QA, Product Owner and finally User to decide if the application behaves correctly. Once you have the correct behavior, you set it in stone by writing a test.

But what about TDD, which advocate writing tests before writing code ? If we look at the big picture, TDD's real added value is not really to respect written requirements, but to ensure the design you are conceiving is testable. TDD Katas and scholar exercises are often toys with clearly defined specifications and simple use case. In the real world, you often iterate on your TDD because you realize the unit test you wrote last week contradicts the integration test you are writing today. Because API has changed, because result was not the good one after all, because user change its mind and prefer logging an error than throwing an exception, etc... And it's not an issue. As you wrote test first, you ensure your design is testable now, but also in the future. Even if requirement change, you can easily update your test assertions to match the new expected result.

In some cases, you don't even know what result you would expect until you have implemented your algorithm. This often happen when dealing with scientific computation. Let's imagine this not-so-fictive conversation between a biologist and a developer:

— Hi, I want to compute size of a Coronavirus from CryoEM acquisition.
— Ok, do you have a dataset ?
— Sure, here it is.
— Cool, and what is the expected size of virus in this dataset ?
— Are you kidding me ? That's what I'm asking for !

So you go home with inputs data but no outcomes. You start writing a test, because you know TDD is a good practice, but you don't know what to write as an expected result. Never mind, you leave it blank and start implementing the algorithm. Once you have a result, you give a pre-version to your user which will decide if the result is coherent or not (it often happens in scientific domain that you don't know what result to expect but can validate a result a posteriori). If the result is OK, you put it as expected result in a test to forbid regression in the future.

What is also important to understand is that failing test does not always indicate something is broken. It indicates something has changed. If you make an improvement in your algorithm, the test will fail, indicating the result is different from before. Scientist will tell you if result is better or worst, but if result is better, you have to update test baseline to freeze this new behavior.

In the previous example, you may argue that only the final result is really validated. You cannot guaranty there are no bugs in intermediates results. You just hope that, as the final result is good, intermediate result should be good, no ? Unfortunately, no. And that's lead me to a statement I discover with pain in my career:

Any working software contains an even number of internal bugs which correct each other.

And worse than that, some unit tests are here to ensure these buggy behavior stay in place to avoid collapsing the whole system.

Gross value a net value

Now we have seen what is a test, let's talk about its value.

Like for your salary, gross value doesn't matter that much. What matter in the end is net value. And like your salary, tests have a gross value and a net value. Gross value is the test ability to detect regression. Net value is much more subtle.

A test failing half of the time for no reason, taking several minutes to execute and a day to be deployed on the developer environment has a negative net value. You waste more time maintaining it than you actually gain by catching the regression early. On the other hand, a test executed on a few seconds, both on test and developer environment, which detect only true positive is much more valuable.

To have a high net value, a test should:

Have a high probability of detecting regressions
Have a low probability of creating a false positive
Be fast and easy to execute, both on Test and Dev environment
Have a low maintenance cost

Filter your test

Choosing if a test is valuable or not really depend on your domain. Critical DO-178A aeronautics system should be fully tested, whatever the cost. Temporary mobile applications don't need such a robustness. It is up to you to place the cursor. Then, you (I mean, not you personally, but your organization) have to be honest with yourself and drop the tests you judge not profitable enough. The time you will save by not maintaining useless tests can be invested in writing new highly valuable tests. In a nutshell:

Chevron Anti-pattern

2022-04-20T08:21:00+02:00

I will talk about an antipattern I've encountered a few time, but I didn't find it in literature, so I called it the "Chevron antipattern". Let-me know if you recognize it and if it has a true name 😉

Class Diagram

Let's start by an example. Assume you develop an application that should support 2 render engines. One, let says, based on Unity and the other based on Unreal, for example. Your render area is embedded in a Window. You define a Widget abstraction layer to handle event, and a Render abstraction layer to handle rendering. You end up with this class diagram:

You have your Windows that has a WidgetInterface which can be either a Unity based widget or an Unreal based widget. Nice, isn't it ?

I called it "Chevron" pattern because one can see 2 chevrons pointing to the top.

And this can continue on several layers.

Why is it an Anti-Pattern ?

The antipattern part comes from the RendererInterface. An interface defines a set of services that should be implemented by concrete implementation, so that client of this interface don't care about the underlying implementation.

In this case, RendererInterface is not used by any client.

Even if it is not owned by anyone, it may still be used as a function parameter, but when I encountered this pattern, those usages could easily be removed with small refactoring.

You end up having very strong constraints on how concrete Unity and Unreal Renderers have to be implemented, without real reasons. Maybe UnityWidget is just a pass-through UnitRenderer, but this design forced you to have such a layer.

How to Fix This ?

This antipattern is very easy to fix if discovered soon enough. If RendererInterface is useless, just remove it:

Now it decouple UnityRender and UnrealRender implementations. You can implement those classes as you want, even merge them into the Widget layer if it makes sense. Keep in mind that the important part here is the WidgetInterface. It is this interface that let you change render engine without impacting Windows layer. RendereInterface was initially added, probably to keep a symmetry in the design and having a beautiful class diagram. But this is not a good reason to define an interface.

Scrum retrospective: Cool Runnings

2022-04-19T09:12:00+02:00

If you are in your mid-thirties, you probably remember Cool Runnings (or Rasta Rocket for French people). I watched this movie a dozen times when I was a child. But did you know this movie was about a young Scrum team doing its Agile transformation ? Let's have a look at this.

The team

So we have a group of Jamaican trained for olympic bobsled event leaded by their captain Derice Bannock. They are technically ready but struggle to find their place in the competition.

The winner's secret

Derice, the captain, go to a training of the Swiss team, favorite for the event, to see what is their secret for being such a powerful team.

The Swiss team happens to butt each other's heads before the event to reinforce their team spirit and count "Eins Zwei Drei" when pushing the bobsled. So, Derice decide to hit its teammates head before the qualification race and count "Eins Zwei Drei".

Yeah, you got it: Cargo cult...

But things didn't went as planned and they did a catastrophic departure...

The retrospective

Going back to their room, the team analyze what has failed. Derice don't understand what they did wrong as they did exactly like the Swiss team. Sanka gets angry and explain they are not Swiss, they are Jamaican. There is no point doing a Swiss bobsled, they must do a Jamaican bobsled.

The next sprint

Team finally find its own way of doing with their famous "Cool Running" battle cry.

They qualify for the finals, do an excellent race and won the golden medal.... No, just kidding, they crash just before the finish line.

So What ?

We have a team that were told to do Scrum. So they do Sprint planning, they have a Jira backlog with lots of ticket, they do "Stand up" meeting, but as they were still too long they even tried "Push up" meeting, they do Sprint reviews, Sprint retrospectives, smash their head and count "Eins Zwei Drei" at the beginning of each sprint, but it still doesn't works. Why ? Because a Team has to find its own way of working. All the scrum ceremonies are not the cause of becoming Agile. They are the consequences.

The Cool Runnings retrospective

I designed a few years ago a sprint retrospective based on this movies. I spent a couple of minutes explaining the plot like what I did in this article, and then asked the team to put all the Scrum stuff we were doing into 2 categories:

And something really interesting happened. I had a bunch of grumpy people at this time that put everything in the "Eins Zwei Drei" column. Retro, stand up, planif, etc... When I asked them why they put ticket here, they quickly run out of arguments. In the end, they decided to put the stand up at 10am by themselves, they decided to do continuous backlog refinement along the sprint and have a quick planif on monday morning, they chose their DoR and DoD, they chose their workflow in Jira, etc... So my grumpy team that continuously complained about the process just realized that They put this process in place for reasons, and if it doesn't fit their needs anymore, They have to change it.

How to Evaluate a Refactoring Project ?

2022-04-11T08:53:00+02:00

I won't surprise anyone if I say evaluations are most of the time underestimated. Parkinson's law ensure estimations will never be overestimated anyway, but how can we better anticipate all the impediments that will delay our project ?

This article is taken from the "How (not) to refactor a kernel" conference I did during the Agile Tour 2021 in Bordeaux.

Problems occurs at the end

The devil is in the details, and details happen at the end. For a lot of project, startup goes well. You put in place architecture, you handle generic case, post-pone all the small features that bother you and are confident you will meet the deadline. But approaching 80% of project scope, you start to slow down, corner cases don't fit in your design, more and more cases were not anticipate and in the end, you end up with another 80% of work to address and your burn up curve look like this:

Modeling the curve

"The more there is to do, the easier it is to pick stuff to do"
"When nothing works, small changes have big impact"
"The last use cases are the hardest to address"

I think I'm not the only one to already have heard this regarding project advancement. Lets use these statements as starting point for our model. Note that I cannot prove they are true. I suppose they are true and I will extrapolate what they imply. You are free to disagree with them and thus, disagree with the the conclusion I'm going to draw.

Assume we have a finite and constant quantity of work to address in order to fulfill our project. This can be a number of use case, a number of bugs, a performance level to reach, etc... As we stated before, the greater is this remaining quantity, the more effective the team will be at addressing it. This can be formalized by this formula:

For a small period of time (let say, a Sprint), the work done by the team depends on the remaining work, using a constant coefficient "effort" which characterizes the team. We assume the team is constant and stay focus on its project, so the effort coefficient is constant. Now, lets do some math and write this equation slightly differently with infinitesimal time:

You may recognize a 1st order differential equation which is solved this way:

With k and τ constant. Their values is not meaningful for our demonstration. If we plot this equation, we end up with this curve:

This curve really look like the project burn up curve we had at the beginning, so our model may not be that silly after all.

Graphical interpretation

1st order equation have some particular characteristics easily visible graphically. First of all, they never reach their limit. This is annoying to estimate the end of our project... In practice, we consider this kind of system reach its limit when it reach 95% of its target. And we know that these systems take the same time to go from 0 to 80% than to go from 80% to 95%. This means that, once you had implemented 80% of feature, you are just in the middle of your project:

Despite being a very good way to kill the mood of your team, this information is not very useful to build a roadmap because you have to wait the middle of your project to know when it will be finished. But this curve has another much more interesting particularity: Its initial slope tells us when it will reach the target which is actually 3 times latter than you think:

As any human being, as soon as you start having a curve you hasten to draw a straight line to extrapolate its evolution. This extrapolation gives you an estimated end date that you happily communicate to your direction and stakeholders. But in this case, you know your end date will be 3 time this estimation. This can be summarized by:

It will take tree time your initial velocity to complete the project

Is 3 a magic number ?

I've heard some project manager stating that estimation should always be multiplied by 3, or even Pi, using more or less serious justifications like "Three-point estimating technique" or the infamous "Circular Estimation Conjecture". But my favorite argument for Pi is that, as an estimation is always irrational, multiply it by Pi will ensure making it irrational anyway.

The Problem of Floating Point Precision in Opengl, Vulkan and 3D in General - Part 3

2022-04-04T10:38:00+02:00

We have seen in the previous article how to handle position of object and camera in very large scene. But this still won't solve the shaky problem caused by matrix accumulation. We will here see that only a few place needs to use double precision, and not all your rendering pipeline.

Reminder about space changes

We have a point P expressed in Model space, and we want to express it in View space knowing Model matrix and View matrix:

Model is a shortcut for ModelToWorld matrix. With this matrix, you can express P coordinates in World space knowing P coordinates in Model space. On the other hand, View matrix is a shortcut for WorldToView matrix with which we can express P coordinate in View space knowing P coordinate in World space. In the end:

P_view = WorldToView * P_world = WorldToView * ModelToWorld * P_model

The (WoldToView * ModelToWorld) is also known as the ModelView matrix. Legacy OpenGL only used the ModelView matrix for performance reason and didn't split it into 2 Model and View matrix, but separate World and View are much more convenient, especially for beginners. And in todays computer, multiplying 2 matrices together is really not a bottleneck anymore.

What is P precision in view space ?

As we saw, precision issues occurs when point of interest is faw aways from its reference space. Both Camera and P are far away from World origin. We saw in previous article how to handle that. But when Camera and P are really close to each other, why P would be shaky ?

And once again, precision comes into play...

Problems are located in 3 place: the WorldToView matrix, the ModelToWorld and the WorldToView * ModelToWorld**** computation. Imagine P and Camera located in Saturn with an Heliocentric world space. You will do a +1 billion km to express P in World space and -1 billion to go back to View space. Now you slightly move your camera of 1 meter to the left and your P of 1 meter to the right. Using a proper model space and tripod position as described before, you can do it. You do 1 000 000 000 001 - 999 999 999 999. Cast these values to Float to generate corresponding matrices and your meter has been lost on both side. So should we accumulate everything in double precision ?

Well, not exactly. Only a few thing should be done in double precision. Most of other space transformations can remains in single precision.

Managing model matrix

Render engines often have a stack of several transformation before reaching the final object. To render a tire on car on a track, you have the main car position, then translation from car origin to tire origin, then rotation of tire around its axis. For complex models, you may have a dozen of transformation like this before reaching the actual point. But not all transformation should be done in Double. Car may be far far away from its origin, like Elon Musk's Tesla Roadster, but once on the car, object are relatively close to each other. So only the first big translation should be accumulate in double:

If we represent this scene with a scene graph, we would have 3 Translation node T1, T2 and T3. T2 and T3 can use float precision and matrix accumulation can be done in float. Lets call T'=T2*T3.
However, even though T1 translation value can be stored in float, the computation T"=T1 * T' should be done and stored in double precision and final Model matrix result stored in double precision. In the end, this scene graph should look like this:

A root node, immediately followed by a "Double translation" which generate a Double precision Model matrix, followed by classical scene graph: translations working in single Float precision and final object with single precision coordinated.

Managing View matrix

Same story for the view matrix. You should first locate the camera "Tripod" using a double precision translation and then locate your camera as usual:

So finale scene graph look like this:

And what about ModelView ?

In the end, we end up with a double precision Model matrix and a double precision View matrix. With them, we can have the final double precision ModelView matrix. And you will tell me "Yeah, but GPU don't work with double precision, they need single precision ModelView matrix to compute all the vertices, lighting and so on".

Yes, you are right. We cannot send double precision ModelView matrix directly to GPU. But suddenly, magic comes into play...

Remember, ModelView express coordinates of point relative to Camera. In our case, both point and Camera are somewhere around Saturn, they are both close to each other and thus, we don't need a lot of precision to locate a point relative to camera. We don't care anymore about World space. Remember this diagram:

This means, once we have computed ModelView = View * Model in double precision, we can cast back ModelView matrix to single precision and send it to GPU. ModelView matrix no more have large magnitude coefficient and precision won't be an issue anymore.

Difficulty is how to define the initial double precision translation. Unfortunately, it depend on your use case. This big translation can be seen as a local origin for your scene, so for wide open world, this can be the tile position where you currently are. For fly simulator, you can update this translation with current plane position every hundred of km for examples. I had a use case where user wanted to render different factories in a Google earth like background. Off course, each factories were very detailed and you can literally see the pipes and ladders when zooming it. In this case, local origin was the position of currently selected factory.

The Problem of Floating Point Precision in Opengl, Vulkan and 3D in General - Part 2

2022-03-28T07:56:00+02:00

We have seen in a previous article why floating point representation may shake our objects in scene. We will now see how to use space change tricks to mitigate this problem.

A brief reminder about World, Model and View space

When dealing with 3D coordinates, we generally use 3 coordinate systems: World space, Model space and View space. World space is stuck on the ground. Model space is stuck on the object. So there are lots of model spaces in you scene, one for each object. View space is stuck to the camera. I won't talk about Projection space, which define camera lens, in this article.

Coordinates of P are always relative to a space. So here, coordinates of P relative to Model space are (1, 0, 1). But we can Also define coordinates of P relative to other spaces:

P_model = (1, 0, 1)
P_view = (-1, 0, 3)
P_world = (4, 0, 5)

What about precision of those coordinates ?

One can see that P_world have a higher magnitude than P_model, because P is far away from World space origin. Well, not so much in this exemple, but imagine Model spaces millions miles away from World Space. What happen when taking in account floating precision error ?

P will actually jump to a floating representable position. But what happen if we locate P using P_model instead ?

P would jump to a much closer position. So close that you would barely see P jumping when moving it relative to model space. However, you may have noticed that model space has to be aligned into world space representable values. In this case, moving model space will make it jump from one position to another. So to use this trick, you have to define a fixed space close to your point of interest, and move you points relative to this reference space.

What about View space ?

The View space moves a LOT when doing 3D rendering. It is probably the most dynamic space. It can be linked to a car for racing game, player for FPS, mouse for CAD application, etc... So we really have to ensure this space move smoothly if we don't want to give our users a headache. To do so, we can use the same trick than before. Instead of locating camera relative to world space, we can define a "Tripod" space, and locate your camera relative to this tripod:

Isn't it Double precision with extra steps ?

In the end, what we did was encoding a position with 2 floats, giving us 64bit floating precision. But this trick can be implemented using your current render engine architecture. It doesn't require using double precision values in computations, and any render engine already has a "Translation" object that can be used to do this. Last but not least, it doesn't change your current floating precision API.

Unfortunately, for various reasons, Render engines require expressing all the coordinates in View space at some point. Accumulating transformations like we did won't solve the precision error we will have when expressing coordinates in View space. We will see in the next article how to handle this with very local modifications to our matrix accumulation code.

The Problem of Floating Point Precision in Opengl, Vulkan and 3D in General - Part 1

2022-03-21T09:06:00+01:00

When zooming a lot on 3D objects or when having small object lost in a very large scene, it can happen that your objects "jiggle" from one position to another, making rendering really shaky. We will see in this series of article where does it comes from and how to workaround it without using double precision everywhere.

What we would like

Lets think in 2D. We have an X/Y coordinate system. When we specify the (x, y) coordinate of some point, we would like this point to be located at (x, y):

What we have using integer coordinate

What would happen if we were using integer coordinate system ? When locating a point in (1.5, 1.3), its position would be rounded to (1, 1). And if we move this point, it will jump from position to position:

What we have using float coordinates

Floating points suffer from the same issue than integer values. Only a set of position are accessible, the difference is that, near to 0, a lot of position are possible. But the further away from zeros, the further the possible positions are. Instead of having regular grid of position like for integers, we have a logarithmic grid:

What does it means ? It means that for small scene you won't see your point jumping from one position to another. There are enough precision, enough possible position so that your point correspond to what you expect. But for large scene, object far from the origin will not have a lot of possible position and will jump from one point to another.

What does "large scene" means ?

I'm talking about "large scene", but what does it means practically ? As a first approximation, you can consider your scene as large if you need more that 6 digit precision to locate your objects. I will detail why 6 digit later.

So a large scene does not depend on the absolute size of your scene but the ratio between your smallest object and the whole scene. A solar system can be consider as a small scene: Distance from Earth to the Sun is around 150 millions km, or 150.10e6 km. Meaning with 6 digit precision, assuming heliocentric coordinates, you can locate the earth position +/- 100km in space. If your goal is just to have an overview of the main planets in space, then this precision is enough. No one will notice earth is actually 100km to the left when looking at the whole system.

However, if you want to localize the International Space Station which orbit at ~400km altitude, and zooming on it, then 100km precision is not enough. You cannot just use heliocentric coordinates with single floating point precision. When animating your station, it will jump from one position to another.

Where does these "6 digits" comes from ?

Computers does not have infinite amount of memory, and thus cannot store infinite amount of digits. Furthermore, they work only with power of 2, but lets thinks with decimal system for now.

Base 10 precision

To represent a floating value using finite precision, you can store a certain amount of digit times a power of 10. For example, to store the value "1.5", you will store the value "000015 * 10^-01". In this case, 000015 is called "the fraction" and ^-1 is called "the exponent".

Here is a few other example:

150 => 000015 * 10^01
150 000 000 000 => 000015 * 10^10
0.15 => 000015 * 10^-02

I think you get the point. Values are stored as M * 10^n, with M having a fixed size of 6 digits and n an exponent of size 2.

But what happen when using more than 6 digit precision ?

123 456 789 => 123456 * 10^3 => 123 456 000
0.123 456 789 => 123456 * 10^-9 => 0.123 456 000

You will just "drop" them, as it cannot store them in our representation. Note that exponent is also clamped. As we cannot store more than 2 digits for the exponent, we cannot store value higher than 999999 * 10^99.

Base 2 precision

Now, lets go back to power of 2. As you know, computer use base 2 representation, not base 10. Number of base 2 digit they can store is defined by the IEEE 754 standard, which gives for 32bit floating point type:

1 bit for sign (>0 or \<0), 8 bits for exponent (so up to ^255) and 23 digit. But where does the "6 digit in base 10" comes from ? To understand it, you have to ask "how many base 2 digits is required to represent a base 10 digit" ? The answer is:

So, 23 base 2 digits can represent ~6.9 base 10 digits:

It is not easy to understand what is "6.9" digits after all. A method I use is that, as a very unprecise approximation, you need 3 bits to count up to 10 (111 = 7 ~=10). 3 binary digits represent 1 decimal digit. So 23 binary digit represent 23/3=7.6 decimal digit. As we know this is a very rough approximation, lets get safe and take only 6 decimal digit precision.

Using Double precision

Double precision are like single Float precision, except they use 11 bits for exponent and 52 bits for fraction, which can represent ~15 decimal digits precision instead of 6.

But using double precision has a cost regarding memory consumption, performances and legacy code refactoring effort.

I hope you better understand where does this problem comes from. I will develop in next articles the problem of floating precision when accumulating transformation matrices.

The problem of floating point precision in OpenGL, Vulkan and 3D in general - part 2

An Overview of Some Git Branching Models

2022-03-14T09:27:00+01:00

Git is a wonderful tool. But like all tools, it require methods to be used correctly. We will cover in this article some git branching model I have encountered during my career.

Trunk Only Development

Used either by student who don't know what they are doing or by skilled and mature team as an extreme interpretation of "Trunk Based Development".

Pros:
- Real time integration.
- Makes developers aware of their responsibilities: If you break something, everyone is impacted.

Cons:
- When a commit break something, everyone is impacted...
- Don't forget to pull before pushing.
- Quickly becomes a real mess if your team is not mature enough. Even if history stay linear, your product will be broken several times a day and developers will spend lot of time fixing conflict.

One branch per developer

If you don't know what to do, take this one. Simple to explain, very well adapted to Github's Pull Request or Gitlab's Merge Request paradigm, and ensure some stability even with beginners team.

Pros:
- Master branch always works.
- No integration dependency. Each developer is responsible of its own branch.

Cons:
- Isolate developer on its own task.
- No validation of branches together.
- Require well dimensioned test infrastructure. May not be possible for embedded development or heavy legacy software.

One branch per feature

Similar to the one before, but enforce collaboration between developers. For me, the best trade off between having real time integration and having a stable Master branch.

Pros:
- Master branch always work.
- No integration dependency.
- Several developers can work on the same feature.

Cons:
- No validation of branches together.
- Require good collaboration between developers.
- Require well dimensioned test infrastructure. May not be possible for embedded development or heavy legacy software.

Trunk Based Development

This one seems to be the most fashionable model today. Warning, "Trunk Based Development" is not "Trunk Only Development". Developers can work both on master branch for small fixes or on feature branches for more complex development, as long as feature branches have a short lifetime. You can take a look to this article to have more details about this model.

Pros:
- Benefit of real time integration and longer term development
- Empower collaboration and accountability.

Cons:
- Master can be broken
- High and variable pressure on test system => Require a dynamic scalable infrastructure.
- Risk of drifting to Trunk Only Development by laziness.

Nightly integration test

Developer merge their branch in a "Nightly test branch". Test are executed nightly and result are available in the morning. If you don't have a very exotic test system with strong constrains that prevent you doing real continuous integration, please don't do this.

Pros:
- Branch are validated together.
- Load is constant on test infrastructure.
- Can handle complex test system or very long tests.

Cons:
- Have to wait the next day to have results.
- If test infrastructure crash during the night, have to wait another day.
- When test fail, have to investigate which branch is responsible.

Gitflow

Gitflow has a little success back in the 2015's, especially with the git-flow CLI tool. It is today more and more criticized for its complexity and the latency it introduced in integration. What I personally think about Gitflow can be summarized by:

Just because you are doing something complicated doesn't mean you are doing something smart.

Pro:
- Explicitly handle all the case than can occurs during product life time: Feature development, stabilization, code freeze, last minute hotfixes.
- Each branches are tested before integration and common result is tested before release.

Cons:
- Complicated to explain for people who are not familiar with Git.
- Lot of branches, lot of complexity, lot of mistakes.
- Going from feature to master takes times, slowing down continuous integration.

Guitar hero world tour

Pro:
- Very good with friends

Cons:
- Is not a branching model

How to Justify Refactoring to Your Product Owner ?

2022-03-07T13:57:00+01:00

Refactoring is an intrinsic part of software development. However, it seems often difficult to justify a refactoring to the Product Owner. We will see in this article how to properly justify refactoring, when it is useful but also when refactoring is a lost of time.

Why refactoring is a good thing ?

Some developer see refactoring as a failure, especially when they have to refactor their own code. A failure of having conceived a good evolutive design. A failure for having chosen the wrong architecture. But it is not the case. You have conceived a software with the vision, needs and skills your had at that time. Now you have a better vision of how things should works. Now you have new needs that you couldn't anticipate. Now you are a better developer. Now you know what is good and what is wrong with current design, and you are ready to improve your code. So let's state a point:

A good design is not a design that won't need refactoring in the future. A good design is a design that will facilitate refactoring in the future.

So refactoring is mandatory to make your code follow your business, to make your code evolve in the direction you chose. Refactoring is that good friend that drive its drunk buddy until its home and avoid it being lost in the middle of the highway.

Why refactoring is seen as a lost of time ?

Let me be the devil's advocate to see why some Product Owners may be reluctant to do refactoring. From a product point of view, refactoring don't add any value. You begin with a set of feature and you end up with the same set of feature with the risk of breaking something.

— Yeah, but the code is cleaner !
— Customer don't care about code.
— Yeah, but it is less buggy !
— By how much ? Do your refactoring will fix bugs we have in our database ? Which one ? We don't care about 5 years old bugs, we care about important bugs that has been opened recently. How many bug will it fix compared to bug it will create ?
— Yeah, but future feature will be faster to implement !
— By how much ? You want to plan a 1 month refactoring. Could you ensure that future feature will take half the time to be done so that it can be profitable in 2 months ?

I think you got it. Refactoring for refactoring is pointless and risky. "The code will be cleaner" is not an argument. How can you prove that YOUR code is actually better than existing code ? And what does it mean to have a "clean code" ?

How to justify refactoring ?

Refactoring is a good practice but it should be done intelligently. You must have a reason to touch existing code and you should do it to achieve a goal. I will present below some good reasons to refactor a code:

Refactor to implement a story

Never say to your PO: "We can do a quick and dirty implementation for 2 points but this would require 5 points to do it correctly". Your PO will understand "We can do it for 2 points or for 5 points". Just tell him this story is 5 points, and use this time to do it properly. Remember The Boy Scout Rule: "Always leave the campground cleaner than you found it." But keep in mind the main goal is the story, not the refactoring.

Refactor to enable more features

Sometime you really need a 10 point refactoring effort to implement a 1 point story. In this case, you should ask yourself: does this effort will enable other 1 point stories ?

If the answer is no, then is the effort worth it ? Maybe this development touch an old forgotten code that probably won't be modified in the next 10 years. In this case, maybe it is better to do a small hack and let this code rest in peace.

But if the answer is yes, you should show to your PO what features this refactoring will make possible and not focus on the beauty and readability of the new code. PO don't care about code.

Refactor to improve metrics

The 2 previous parts were focused on story oriented refactoring. A user story trigger addressing technical dept. But sometime, you want to rewrite a part of code just because it is really hard to maintain and you've had enough working in dirty code. In this case, you must show, with facts, that new code is actually better than previous one. Here is a non-exhaustive list of arguments that demonstrate a refactoring was actually useful :

You have significantly decreased line of code.
You have replaced custom code by well established 3rd part library or tool.
You have removed a dependency with an annoying/unsupported 3rd part library or tool.
You have improved code coverage (or any code quality metric).
You have decreased build/test/packaging time.

If the only argument you have is "The code is cleaner and more readable", then you didn't refactor your code. You have only written different code. Not better, not worse than before. Just different. And you now are the only one who know how it works, congratulation.

KissYagni

Quantum Processing – Creating Quantum circuit with qiskit.

Installing qiskit

First program

Bernstein-Vazirani Algorithm

Bernstein-Vazirani Classical Algorithm

Bernstein-Vazirani Quantum Algorithm

Takeaways

The Deutsch algorithm.

The Query Model

The Deutsch Problem

The Query Gate

The Deutsch Algorithm

\(\left| \Psi_0 \right>\)

\(\left| \Psi_1 \right>\)

\(\left| \Psi_2 \right>\)

\(\left| \Psi_3 \right>\)

Takeaways

Quantum Processing – The Controled Gates.

Conservation of quantum information

Representing multiple qbits

Controlled gates

CNOT Gate

Controlled Y Gate

Controlled Z Gate

Takeways

Quantum Processing – The Quantum Gates: Single QBit.

Small reminder of classical gates

QBit Vector Representation

Identity

Pauli X

Pauli Y

Pauli Z

Hadamard

Takeaways

An introduction to Quantum Processing - The QBit

Quantum Superposition

Vector representation

The Bloch sphere

Takeaways

Forced Perspective in Video Game

Where to put the Camera ?

Forced perspective

Synergia's Forced Perspective

Conclusion

Evolutive Design: A new approach to conceive complex system.

Darwin and the theory of evolution

Predators are drivers of evolution

What does all this have to do with software engineering ?

Tests are software predators

Test Driven ~~Development~~ Design

Tests prevent Design regressions

Refactoring is not a Crime

Conclusion

How to Profile Unity WebGL Based Application on Your Browser ?

Using Chrome profiler

Enable Symbols Generation

Enable Symbols Generation !!!

Profiling App

How to have your own ChatGPT on your machine (and make him discussed with himself)

Installing GPT4ALL on your machine

What is a Model ?

Going back to GPT4All

Using GPT4All in Python

Conclusion

How to do Graphical Tests in Unity ?

Introduction

Graphics Tests Framework

Adding a new Graphical Test

Conclusion

How to Handle Double Precision Coordinates in Unity ?

The Unity High Precision Framework

Using The Unity High Precision Framework

The HPRoot component

The HPTransform component

Understanding the Problem of Float Precision

Strategies to Update HPRoot

Limitations

Conclusion

How to Design a Good API ?

Test Driven Development Design