binarybuffer: Fix str_to_buf() parsing function

The function str_to_buf() was too benevolent and did
not perform sufficient error checking on the input
string being parsed. Especially:

- Invalid numbers were silently ignored.
- Out-of-range numbers were silently truncated.

The following commands that use str_to_buf()
were affected:

- reg (when writing a register value)
- set_reg
- jtag drscan

This pull request fixes that by:

- Rewriting str_to_buf() to add the missing checks.
- Adding function command_parse_str_to_buf() which can
  be used in command handlers. It parses the input
  numbers and provides user-readable error messages
  in case of parsing errors.

Examples:

jtag drscan 10 huh10

- Old behavior: The string "huh10" is silently
  converted to 10 and the command is then executed.
  No warning error or warning is shown to the user.
- New behavior: Error message is shown:
  "'huh10' is not a valid number"

reg pc 0x123456789

Assuming the "pc" is 32 bits wide:

- Old behavior: The register value is silently
  truncated to 0x23456789 and the command is performed.
- New behavior: Error message is shown to the user:
  "Number 0x123456789 exceeds 32 bits"

Change-Id: I079e19cd153aec853a3c2eb66953024b8542d0f4
Signed-off-by: Jan Matyas <jan.matyas@codasip.com>
Reviewed-on: https://review.openocd.org/c/openocd/+/8315
Tested-by: jenkins
Reviewed-by: Marek Vrbka <marek.vrbka@codasip.com>
Reviewed-by: Antonio Borneo <borneo.antonio@gmail.com>

src/helper/command.c

@@ -1360,6 +1360,46 @@ int command_parse_bool_arg(const char *in, bool *out)
 	return ERROR_COMMAND_SYNTAX_ERROR;
 }
 
+static const char *radix_to_str(unsigned int radix)
+{
+	switch (radix) {
+		case 16: return "hexadecimal";
+		case 10: return "decadic";
+		case 8: return "octal";
+	}
+	assert(false);
+	return "";
+}
+
+COMMAND_HELPER(command_parse_str_to_buf, const char *str, void *buf, unsigned int buf_len,
+	unsigned int radix)
+{
+	assert(str);
+	assert(buf);
+
+	int ret = str_to_buf(str, buf, buf_len, radix, NULL);
+	if (ret == ERROR_OK)
+		return ret;
+
+	/* Provide a clear error message to the user */
+	if (ret == ERROR_INVALID_NUMBER) {
+		if (radix == 0) {
+			/* Any radix is accepted, so don't include it in the error message. */
+			command_print(CMD, "'%s' is not a valid number", str);
+		} else {
+			/* Specific radix is required - tell the user what it is. */
+			command_print(CMD, "'%s' is not a valid number (requiring %s number)",
+				str, radix_to_str(radix));
+		}
+	} else if (ret == ERROR_NUMBER_EXCEEDS_BUFFER) {
+		command_print(CMD, "Number %s exceeds %u bits", str, buf_len);
+	} else {
+		command_print(CMD, "Could not parse number '%s'", str);
+	}
+
+	return ERROR_COMMAND_ARGUMENT_INVALID;
+}
+
 COMMAND_HELPER(handle_command_parse_bool, bool *out, const char *label)
 {
 	switch (CMD_ARGC) {