Wednesday, September 9, 2026

Testing an Azure Communication Services SMTP + XOAUTH2 connection

I spent a while debugging an SMTP XOAUTH2 setup against Azure Communication Services that kept failing inside another application, without much visibility into what was actually happening on the wire. Pulling the same credentials out into a small standalone script made the problem obvious pretty quickly. Sharing it here in case it saves someone else the same loop.

Before you start, you need:

  • An Azure Communication Services Email resource with a linked, verified domain
  • A Microsoft Entra app registration granted the Communication and Email Service Owner role (or a narrower custom role) on the Communication Service resource
  • An SMTP Username resource created under that Communication Service, linking the Entra app to a chosen username (az communication smtp-username create, or the Portal's "SMTP Usernames" blade)
USERNAME and FROM are not the same thing. SMTP_USERNAME is the identity the SMTP session authenticates as — the value you set with az communication smtp-username create, linked to your Entra app. FROM_ADDRESS is the actual sender address on the message, and Azure only accepts addresses from a pre-approved list per domain, regardless of what SMTP_USERNAME is set to. To stay safe, use DoNotReply@<your-domain> for FROM_ADDRESS unless you've explicitly added another sender username in the Portal under Email → Sender Usernames. See this Microsoft Q&A thread for the full explanation and also from https://davebarr.dev/550-5.3.5-email-senders-username-is-invalid/.

Fill in the CONFIGURE block near the top, then run it with the client secret passed in as an environment variable rather than pasted into the file:

import base64
import json
import os
import smtplib
import urllib.parse
import urllib.request

# --- CONFIGURE ---
TENANT_ID = "<your-entra-tenant-id>"
CLIENT_ID = "<your-entra-app-client-id>"
CLIENT_SECRET = os.environ.get("ACS_CLIENT_SECRET", "")

# The literal value you set as --username when creating the SMTP Username resource.
SMTP_USERNAME = "<your-smtp-username-configured-in-azure>"

# Must be a pre-approved sender address for the domain - see the note above.
FROM_ADDRESS = "DoNotReply@<your-domain>"

TO_ADDRESS = "<recipient-address>"
# --- END CONFIGURE ---

SCOPE = "https://communication.azure.com/.default"
SMTP_HOST = "smtp.azurecomm.net"
SMTP_PORT = 587


def get_access_token() -> str:
    if not CLIENT_SECRET:
        raise SystemExit("Set the ACS_CLIENT_SECRET environment variable before running this script.")

    token_url = f"https://login.microsoftonline.com/{TENANT_ID}/oauth2/v2.0/token"
    data = urllib.parse.urlencode(
        {
            "client_id": CLIENT_ID,
            "client_secret": CLIENT_SECRET,
            "scope": SCOPE,
            "grant_type": "client_credentials",
        }
    ).encode()

    request = urllib.request.Request(token_url, data=data, method="POST")
    request.add_header("Content-Type", "application/x-www-form-urlencoded")

    with urllib.request.urlopen(request) as response:
        payload = json.loads(response.read())

    return payload["access_token"]


def send_test_email(access_token: str) -> None:
    auth_string = f"user={SMTP_USERNAME}\x01auth=Bearer {access_token}\x01\x01"
    auth_b64 = base64.b64encode(auth_string.encode()).decode()

    server = smtplib.SMTP(SMTP_HOST, SMTP_PORT)
    server.set_debuglevel(1)  # prints the raw wire-level SMTP conversation
    server.ehlo()
    server.starttls()
    server.ehlo()

    code, resp = server.docmd("AUTH", "XOAUTH2 " + auth_b64)
    print("AUTH result:", code, resp)

    if code == 235:
        msg = f"From: {FROM_ADDRESS}\r\nTo: {TO_ADDRESS}\r\nSubject: XOAUTH2 test\r\n\r\nTest message."
        send_result = server.sendmail(FROM_ADDRESS, [TO_ADDRESS], msg)
        print("Send result:", send_result if send_result else "accepted, no errors")
    else:
        print("AUTH failed - stopping")

    server.quit()


if __name__ == "__main__":
    token = get_access_token()
    send_test_email(token)

A couple of things that tripped me up, in case they help:

  • Authentication succeeding does not mean the send will succeed. AUTH and MAIL FROM are checked separately, and the errors from each look completely different.
  • If you see 550 5.3.5 Email sender's username is invalid, it's almost always the FROM_ADDRESS, not SMTP_USERNAME — switch to DoNotReply@<your-domain> before assuming your DNS or domain setup is broken.

Official docs for the setup steps: Set up SMTP authentication for sending emails - Azure Communication Services

Tuesday, March 20, 2018

Running your app on a physical device for Android

This applies to both Android Studios and NativeScript.

In Android Studios if you cannot see your device in the dialog when trying to run on physical device:


and in NativeScript after running the below command you get:

tns devices

Connected devices & emulators
Searching for devices...
iTunes is not installed. Install it on your system and run this command again.
Cannot find connected devices. Reconnect any connected devices, verify that your system recognizes them, and run this command again.

Try the following:
- Ensure your phone is plugged into the USB
- Enable devloper mode by going to Settings > About phone and tap Build number seven times.
- Enable USB debugging
- Open up the device manager and if there is an unknown ADB interface then:

- Download the Google USB drivers following the instructions here. The drivers should be located at android sdk\extras\google\usb_driver\
- In the device manager right click on the ADB Interface and choose update driver and choose to manually install it by navigating to  sdk\extras\google\usb_driver\
- Pick Android ADB Interface. If everything installs fine your device manager should contain the driver

- Your phone will then ask you to accept the RSA fingerprint from your computer. Click Ok and you should now be able to run your app on your physical device.

Nativescript application running on Android 7.0 throws SSLHandShakeException

The application I am creating has a back-end web API hosted with Node and Nginx. I used Let's Encrypt to create an SSL certificate. The application has a web and mobile front-end. The web side of things all worked fine on the desktop and on the mobile browser. The mobile native application is created with NativeScript.

While testing against an Android 7.0 device I kept getting errors and could not connect to the back-end web API. After discovering NativeScript Playground in particular this sample which I modified to make a call to my web API, I was able to see that the application was throwing a SSLHandShakeException.

javax.net.ssl.SSLHandShakeException: Handshake failed 
at ZoneAwareError at onRequestComplete 
at Object.onComplete 
Initially I thought the issue was due to using Let's Encrypt however after further digging and hunting I came across this StackOverFlow post which suggests "changing the nginx configuration by removing secp384r1 or replacing it with the default (prime256v1)". It turns out that this is a "known regression in Android 7.0, acknowledged by Google and fixed sometime before the release of Android 7.1.1".

To over come the problem I had to change my Nginx config to use:

ssl_ecdh_curve prime256v1;
instead of

ssl_ecdh_curve secp384r1;

Also to support previous version of Android you will need

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

Wednesday, January 25, 2017

Nhibernate getting SQL from HQL

In a previous post I put up some code on how to retrieve SQL from HQL. At the time I did not need to use it, however, now I do need to use it and came across some limitations. The code does not take into account any query parameters.
The project I am working on at the moment allows users to build their own queries using an interface similar to Ms Access. The query builder then converts the UI elements into an HQL statement. The statement could contain group by's and distinct commands which make it quite hard to do paging. I did come across this solution, however; it does not allow me to take advantage of the query caching ability of Nhibernate so that you get the total count once and then always get the next page of data.

The simplest solution to get the total count was to execute a SQL count query with a subquery.

Select count(*) from (.... the main query.....)

To get the main query I needed to convert the HQL into SQL. After hunting around the following was the best I could do with the restrictions I had. This is just the gist of the code after simplifying it so that it doesn't wrap too much.

public class HqlToSqlQueryConverter : QueryImpl
{
  public static HqlToSqlQueryConverter Create(IQuery query, ISession session, IDictionary<String, object> queryParameters)
  {
    if (!(query is QueryImpl))
 throw new InvalidOperationException("This class only return SQL for HQL");

    var sessionImp = session.GetSessionImplementation();
    var queryExpression = new StringQueryExpression(query.QueryString);
    var plan = sessionImp.Factory.QueryPlanCache.GetHQLQueryPlan(queryExpression, false, new Dictionary<string, IFilter>());

    var translationQuery = new HqlToSqlQueryConverter(query.QueryString, sessionImp, plan.ParameterMetadata);

    //Copy the parameters across
    foreach (String name in query.NamedParameters)
    {
      translationQuery.SetParameter(name, queryParameters[name]);
    }
    return translationQuery;
  }

  public String GetSQL()
  {
    var namedParams = this.NamedParams; //returns a copy
    var queryExpression = ExpandParameters(namedParams);
    var queryParameters = GetQueryParameters(namedParams);

    IQueryPlan plan = new QueryExpressionPlan(queryExpression, false, this.Session.EnabledFilters, this.Session.Factory);

    var sqlCommand = plan.Translators[0].Loader.CreateSqlCommand(queryParameters, this.Session);
    var dbCommand = this.Session.Batcher.PrepareQueryCommand(CommandType.Text, sqlCommand.Query, sqlCommand.ParameterTypes);
    String sql = dbCommand.CommandText;
    return sql;
  }
}

This is how I am using it:

      IQuery query = session.CreateQuery(queryString)
                            .SetFirstResult(pageNumber)
                            .SetMaxResults(maxResults)

                            .SetCacheable(true)
                            .Future<object[]>();

      var translationQuery = HqlToSqlQueryConverter.Create(query, session, parameters);
      String sql = translationQuery.GetSQL();

      int posOrderBy = sql.IndexOf("order by");
      if (posOrderBy > 0)
        sql = sql.Substring(0, posOrderBy);


      var count = session.CreateSQLQuery("Select Count(*) as totalCount from (" + sql + ") x")
                        .AddScalar("totalCount", NHibernateUtil.Int32)
                        .SetCacheable(cacheable)
                        .FutureValue<int>();
  
var results = query.List<object[]>();

This only works because I am using Future<>() and the parameters then get sent for both queries together.

Nhibernate SQL scalar query + SetCacheable

Nhibernate throws a cast error when CreateSQLQuery() is used with SetCacheable() and when trying to return a scalar value. The fix for this problem is to use the AddScalar() method. Thanks to this google group message.

    var sql = "Select count(*) as totalCount from myTables";
    var count = session.CreateSQLQuery(sql)
                   .AddScalar("totalCount", NHibernateUtil.Int32)                      .SetCacheable(true) 
                   .FutureValue<int>();

Monday, May 4, 2015

Installing Windows Identity Foundation (WIF) on Windows 8.1

If you are trying to figure out how to install Windows Identity Foundation run-time on windows 8.1 here are the instructions:

"In Windows 8 Windows Identity Foundation is installed (enabled) by turning a windows feature on in Control Panel > All Control Panel Items > Programs and Features > Turn Windows feature on or off the feature is Windows Identity Foundation 3.5. Installers linked in the answer above will not work on Windows 8"

Taken from (http://stackoverflow.com/questions/3001598/where-is-the-microsoft-identitymodel-dll)

If you try to install Windows6.1-KB974405-x64.msu you will get an "Installer encountered an error 0x80096002 The certificate for the signer of the message is invalid of not found".


Sunday, June 29, 2014

Notes on migrating from SQL Server to Oracle with GUIDs and BLOBs

Migrating data from SQL Server 2008 R2 to Oracle 11g XE using Oracle SQL Developer 4.0.2. For testing purposes both database systems are hosted on the same box.

1. First create a connection to the SQL Server database. (See the section "Configuring the Environment" for setting up the third party JDBC drivers)
2. Create a connection to the Oracle system using the "System" account.
3. Create a user account in Oracle for the migration (eg: "migration_repo"):

CREATE USER migration IDENTIFIED BY migration
DEFAULT TABLESPACE users TEMPORARY TABLESPACE temp;

GRANT CONNECT, RESOURCE, CREATE VIEW, CREATE PUBLIC SYNONYM 
TO migration WITH ADMIN OPTION;

GRANT  ALTER ANY ROLE, ALTER ANY SEQUENCE, 
ALTER ANY TABLE, ALTER TABLESPACE, ALTER ANY TRIGGER, 
COMMENT ANY TABLE, CREATE ANY SEQUENCE, CREATE ANY TABLE,
CREATE ANY TRIGGER, CREATE ROLE, CREATE TABLESPACE, 
CREATE USER, DROP ANY SEQUENCE, DROP ANY TABLE, 
DROP ANY TRIGGER, DROP TABLESPACE, DROP USER, 
DROP ANY ROLE, GRANT ANY ROLE, INSERT ANY TABLE, 
SELECT ANY TABLE, UPDATE ANY TABLE 
TO migration;

(Note: the above was taken from this tutorial.)

4.Create a connection to the newly created user account. Right click the connection and choose Migration Repository -> Associate Migration Repository. If the following error appears "try deleting repository before creating repository" then create a new TableSpace and associate the migration account to the TableSpace. This error sometimes occurs when the TableSpace associated with the user is full (see here for details).

CREATE TABLESPACE <tablespace_name>
DATAFILE 'datafile_name.dbf'
SIZE 20M AUTOEXTEND ON;

5. Open the SQL Server connection and right click on the database to migrate. Choose "Migrate to Oracle".
6. Follow the wizard as show in the Youtube tutorial given here. You will need to setup a migration script directory to store all the scripts. This directory will be used in later steps and will be referred to as "migration script directory".
7. In the "Convert" stage of the wizard change the UNIQUEIDENTIFIER mapping from CHAR[36] to RAW[16]

8. In the "Translate" stage only ALL_CONSTRAINTS, ALL_VIEWS and ALL_PROCEDURES were chosen.

9. Click finish to generate the migration scripts and the bottom window panel should list the migration project. If this process fails try deleting the migration repository, and try again to associate the migration repository.

10. Right click "Converted Database Objects" and choose "Translate". This will take you back to the wizard but then it will skip some stages to take you to the "Translate" stage.

11. Select the bits that you need translated and click finish. This will try and convert the objects.

12. Right click "Converted Database Objects" again and this time choose "Generate Target". Select the Oracle System account connection created in step 3 as the connection. Click "Finish" to generate the new database.


13. Right click "Converted Database Objects" again and this time choose "Move Data" and set the mode to "Offline". Click "Finish" to generate the move data scripts.


14. For converting the GUIDS in SQL Server to Oracle RAW[16] some script changes are required. Open up Windows Explorer and navigate to the "migration script directory" created in step 6. Open the folder with the migration project name that was specified in the wizard. Open the "datamove" folder and drill down until you find a folder called "control". The "control" folder contains ctl scripts. You will need to modify all the scripts that contain GUID definitions so that they can be mapped to a RAW, this involves removing the "-" symbol in the GUID and passing it to the HEXTORAW function.

So the original text in the file:
organisation_id CHAR(32)

needs to be modified to:
organisation_id "HEXTORAW(REPLACE(:organisation_id ,'-',''))"

Note: If using a text editor such as NotePad++ the following regular expression can be used for the search and replace:
Find: (.*)\sCHAR\(32\)
Replace: $1 "HEXTORAW\(REPLACE\(:$1 ,'-',''\)\)"

15. This step is only applicable if you have BLOB columns. VARBINARY(MAX) fields in MSSQL are mapped to BLOB columns in Oracle. The migration tool dumps out the hex values in VARBINARY fields as text, oracle then imports this text into a temporary CLOB column which it then runs a stored procedure on to convert it to a BLOB. In the control folder find all the tables that may have BLOB columns and set the char(XX) to an appropriate value depending on how large the data is. Note that the column may have been renamed to SQLDEVELOPER_CLOB_X. See example below.

load data
infile 'data/DEMO.BinaryData.dat'
 "str '<EORD>'"
into table DEMO.BinaryData
fields terminated by '<EOFD>'
trailing nullcols
(
  binaryData_id "HEXTORAW(REPLACE(:binaryData_id ,'-',''))",
  version ,
  SQLDEVELOPER_CLOB_3 char(5564262)
)

Locate the "Orcale_loader.bat" file at the same level as the "control" folder and set the readsize parameter to an appropriate value for each table with a BLOB column.

sqlldr %username%/%password%@%tnsname% control=control\DEMO.BinaryData.ctl readsize=104857600 log=log\DEMO.BinaryData.log

16. Open up a command prompt as administrator. Navigate to the dated directory inside the "datamove" folder.
c:\Windows\System32>cd\
c:\>cd <migration script directory>\<project name>\datamove\<dated folder>

17. Run the following command. If the SQL Server database is hosted on the same box then use localhost for the server name. This command will dump all the data in all the tables into a directory called "data" nested within the above folder structure.

MicrosoftSQLServer_data <server name> <username> <password>

18. Run the following command. The service name will be XE for this scenario. This command requires that the command in step 15 has been run successfully, otherwise you may receive an "lfiopn failed for file (*.log)" indicating that the log folder is missing. A log folder contains details on how many rows were successfully imported and how many failed per table.

oracle_loader <service name> <username> <password>