Well,
You can disable the Cursor display in one of the Brushes, before using the Annotate Tool:
Un-check that and then press D, will probably be the fastest…
Or write a small Python script that disables the Cursor, when switching to the Annotate Tool and enables it again when you switch back…
