Don't use assert() for point-on-curve checks.

The stock assert() implementation turns out to be problematic in the
stm32 environment, due to the lack of an output device, which makes
"assert(foo)" equivalent to "if (!foo) abort()", leading to silent
hangs.

We probably ought to reimplement assert() to do something more useful,
but, for now, avoid using it for "impossible" conditions which we do
seem to be triggering anyway, like the occasional point-not-on-curve
errors we get for points we ourselves have picked when testing
multiple ECDSA clients in parallel.  This should never happen, and we
need to figure out what's causing it, but hanging the HSM when it
happens does not help very much.

assert() is somewhat problematic in an embedded environment in any
case, since anything that can go wrong really should have some kind of
recovery action, but in some of the low-probability cases it's far
from obvious what sane recovery action we could possibly take.

1 files changed, 4 insertions, 2 deletions

diff --git a/ecdsa.c b/ecdsa.c
index 04e67b8..1047a97 100644
--- a/ecdsa.c
+++ b/ecdsa.c
@@ -869,7 +869,8 @@ hal_error_t hal_ecdsa_key_gen(const hal_core_t *core,
   if ((err = point_pick_random(curve, key->d, key->Q)) != HAL_OK)
     return err;
 
-  assert(point_is_on_curve(key->Q, curve));
+  if (!point_is_on_curve(key->Q, curve))
+    return HAL_ERROR_KEY_NOT_ON_CURVE;
 
   *key_ = key;
   return HAL_OK;
@@ -1527,7 +1528,8 @@ hal_error_t hal_ecdsa_sign(const hal_core_t *core,
     if ((err = point_pick_random(curve, k, R)) != HAL_OK)
       goto fail;
 
-    assert(point_is_on_curve(R, curve));
+    if (!point_is_on_curve(R, curve))
+      lose(HAL_ERROR_IMPOSSIBLE);
 
     if (fp_mod(R->x, n, r) != FP_OKAY)
       lose(HAL_ERROR_IMPOSSIBLE);