Tuesday, March 20, 2018

Running your app on a physical device for Android

This applies to both Android Studios and NativeScript.

In Android Studios if you cannot see your device in the dialog when trying to run on physical device:


and in NativeScript after running the below command you get:

tns devices

Connected devices & emulators
Searching for devices...
iTunes is not installed. Install it on your system and run this command again.
Cannot find connected devices. Reconnect any connected devices, verify that your system recognizes them, and run this command again.

Try the following:
- Ensure your phone is plugged into the USB
- Enable devloper mode by going to Settings > About phone and tap Build number seven times.
- Enable USB debugging
- Open up the device manager and if there is an unknown ADB interface then:

- Download the Google USB drivers following the instructions here. The drivers should be located at android sdk\extras\google\usb_driver\
- In the device manager right click on the ADB Interface and choose update driver and choose to manually install it by navigating to  sdk\extras\google\usb_driver\
- Pick Android ADB Interface. If everything installs fine your device manager should contain the driver

- Your phone will then ask you to accept the RSA fingerprint from your computer. Click Ok and you should now be able to run your app on your physical device.

Nativescript application running on Android 7.0 throws SSLHandShakeException

The application I am creating has a back-end web API hosted with Node and Nginx. I used Let's Encrypt to create an SSL certificate. The application has a web and mobile front-end. The web side of things all worked fine on the desktop and on the mobile browser. The mobile native application is created with NativeScript.

While testing against an Android 7.0 device I kept getting errors and could not connect to the back-end web API. After discovering NativeScript Playground in particular this sample which I modified to make a call to my web API, I was able to see that the application was throwing a SSLHandShakeException.

javax.net.ssl.SSLHandShakeException: Handshake failed 
at ZoneAwareError at onRequestComplete 
at Object.onComplete 
Initially I thought the issue was due to using Let's Encrypt however after further digging and hunting I came across this StackOverFlow post which suggests "changing the nginx configuration by removing secp384r1 or replacing it with the default (prime256v1)". It turns out that this is a "known regression in Android 7.0, acknowledged by Google and fixed sometime before the release of Android 7.1.1".

To over come the problem I had to change my Nginx config to use:

ssl_ecdh_curve prime256v1;
instead of

ssl_ecdh_curve secp384r1;

Also to support previous version of Android you will need

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

Wednesday, January 25, 2017

Nhibernate getting SQL from HQL

In a previous post I put up some code on how to retrieve SQL from HQL. At the time I did not need to use it, however, now I do need to use it and came across some limitations. The code does not take into account any query parameters.
The project I am working on at the moment allows users to build their own queries using an interface similar to Ms Access. The query builder then converts the UI elements into an HQL statement. The statement could contain group by's and distinct commands which make it quite hard to do paging. I did come across this solution, however; it does not allow me to take advantage of the query caching ability of Nhibernate so that you get the total count once and then always get the next page of data.

The simplest solution to get the total count was to execute a SQL count query with a subquery.

Select count(*) from (.... the main query.....)

To get the main query I needed to convert the HQL into SQL. After hunting around the following was the best I could do with the restrictions I had. This is just the gist of the code after simplifying it so that it doesn't wrap too much.

public class HqlToSqlQueryConverter : QueryImpl
{
  public static HqlToSqlQueryConverter Create(IQuery query, ISession session, IDictionary<String, object> queryParameters)
  {
    if (!(query is QueryImpl))
 throw new InvalidOperationException("This class only return SQL for HQL");

    var sessionImp = session.GetSessionImplementation();
    var queryExpression = new StringQueryExpression(query.QueryString);
    var plan = sessionImp.Factory.QueryPlanCache.GetHQLQueryPlan(queryExpression, false, new Dictionary<string, IFilter>());

    var translationQuery = new HqlToSqlQueryConverter(query.QueryString, sessionImp, plan.ParameterMetadata);

    //Copy the parameters across
    foreach (String name in query.NamedParameters)
    {
      translationQuery.SetParameter(name, queryParameters[name]);
    }
    return translationQuery;
  }

  public String GetSQL()
  {
    var namedParams = this.NamedParams; //returns a copy
    var queryExpression = ExpandParameters(namedParams);
    var queryParameters = GetQueryParameters(namedParams);

    IQueryPlan plan = new QueryExpressionPlan(queryExpression, false, this.Session.EnabledFilters, this.Session.Factory);

    var sqlCommand = plan.Translators[0].Loader.CreateSqlCommand(queryParameters, this.Session);
    var dbCommand = this.Session.Batcher.PrepareQueryCommand(CommandType.Text, sqlCommand.Query, sqlCommand.ParameterTypes);
    String sql = dbCommand.CommandText;
    return sql;
  }
}

This is how I am using it:

      IQuery query = session.CreateQuery(queryString)
                            .SetFirstResult(pageNumber)
                            .SetMaxResults(maxResults)

                            .SetCacheable(true)
                            .Future<object[]>();

      var translationQuery = HqlToSqlQueryConverter.Create(query, session, parameters);
      String sql = translationQuery.GetSQL();

      int posOrderBy = sql.IndexOf("order by");
      if (posOrderBy > 0)
        sql = sql.Substring(0, posOrderBy);


      var count = session.CreateSQLQuery("Select Count(*) as totalCount from (" + sql + ") x")
                        .AddScalar("totalCount", NHibernateUtil.Int32)
                        .SetCacheable(cacheable)
                        .FutureValue<int>();
  
var results = query.List<object[]>();

This only works because I am using Future<>() and the parameters then get sent for both queries together.

Nhibernate SQL scalar query + SetCacheable

Nhibernate throws a cast error when CreateSQLQuery() is used with SetCacheable() and when trying to return a scalar value. The fix for this problem is to use the AddScalar() method. Thanks to this google group message.

    var sql = "Select count(*) as totalCount from myTables";
    var count = session.CreateSQLQuery(sql)
                   .AddScalar("totalCount", NHibernateUtil.Int32)                      .SetCacheable(true) 
                   .FutureValue<int>();

Monday, May 4, 2015

Installing Windows Identity Foundation (WIF) on Windows 8.1

If you are trying to figure out how to install Windows Identity Foundation run-time on windows 8.1 here are the instructions:

"In Windows 8 Windows Identity Foundation is installed (enabled) by turning a windows feature on in Control Panel > All Control Panel Items > Programs and Features > Turn Windows feature on or off the feature is Windows Identity Foundation 3.5. Installers linked in the answer above will not work on Windows 8"

Taken from (http://stackoverflow.com/questions/3001598/where-is-the-microsoft-identitymodel-dll)

If you try to install Windows6.1-KB974405-x64.msu you will get an "Installer encountered an error 0x80096002 The certificate for the signer of the message is invalid of not found".


Sunday, June 29, 2014

Notes on migrating from SQL Server to Oracle with GUIDs and BLOBs

Migrating data from SQL Server 2008 R2 to Oracle 11g XE using Oracle SQL Developer 4.0.2. For testing purposes both database systems are hosted on the same box.

1. First create a connection to the SQL Server database. (See the section "Configuring the Environment" for setting up the third party JDBC drivers)
2. Create a connection to the Oracle system using the "System" account.
3. Create a user account in Oracle for the migration (eg: "migration_repo"):

CREATE USER migration IDENTIFIED BY migration
DEFAULT TABLESPACE users TEMPORARY TABLESPACE temp;

GRANT CONNECT, RESOURCE, CREATE VIEW, CREATE PUBLIC SYNONYM 
TO migration WITH ADMIN OPTION;

GRANT  ALTER ANY ROLE, ALTER ANY SEQUENCE, 
ALTER ANY TABLE, ALTER TABLESPACE, ALTER ANY TRIGGER, 
COMMENT ANY TABLE, CREATE ANY SEQUENCE, CREATE ANY TABLE,
CREATE ANY TRIGGER, CREATE ROLE, CREATE TABLESPACE, 
CREATE USER, DROP ANY SEQUENCE, DROP ANY TABLE, 
DROP ANY TRIGGER, DROP TABLESPACE, DROP USER, 
DROP ANY ROLE, GRANT ANY ROLE, INSERT ANY TABLE, 
SELECT ANY TABLE, UPDATE ANY TABLE 
TO migration;

(Note: the above was taken from this tutorial.)

4.Create a connection to the newly created user account. Right click the connection and choose Migration Repository -> Associate Migration Repository. If the following error appears "try deleting repository before creating repository" then create a new TableSpace and associate the migration account to the TableSpace. This error sometimes occurs when the TableSpace associated with the user is full (see here for details).

CREATE TABLESPACE <tablespace_name>
DATAFILE 'datafile_name.dbf'
SIZE 20M AUTOEXTEND ON;

5. Open the SQL Server connection and right click on the database to migrate. Choose "Migrate to Oracle".
6. Follow the wizard as show in the Youtube tutorial given here. You will need to setup a migration script directory to store all the scripts. This directory will be used in later steps and will be referred to as "migration script directory".
7. In the "Convert" stage of the wizard change the UNIQUEIDENTIFIER mapping from CHAR[36] to RAW[16]

8. In the "Translate" stage only ALL_CONSTRAINTS, ALL_VIEWS and ALL_PROCEDURES were chosen.

9. Click finish to generate the migration scripts and the bottom window panel should list the migration project. If this process fails try deleting the migration repository, and try again to associate the migration repository.

10. Right click "Converted Database Objects" and choose "Translate". This will take you back to the wizard but then it will skip some stages to take you to the "Translate" stage.

11. Select the bits that you need translated and click finish. This will try and convert the objects.

12. Right click "Converted Database Objects" again and this time choose "Generate Target". Select the Oracle System account connection created in step 3 as the connection. Click "Finish" to generate the new database.


13. Right click "Converted Database Objects" again and this time choose "Move Data" and set the mode to "Offline". Click "Finish" to generate the move data scripts.


14. For converting the GUIDS in SQL Server to Oracle RAW[16] some script changes are required. Open up Windows Explorer and navigate to the "migration script directory" created in step 6. Open the folder with the migration project name that was specified in the wizard. Open the "datamove" folder and drill down until you find a folder called "control". The "control" folder contains ctl scripts. You will need to modify all the scripts that contain GUID definitions so that they can be mapped to a RAW, this involves removing the "-" symbol in the GUID and passing it to the HEXTORAW function.

So the original text in the file:
organisation_id CHAR(32)

needs to be modified to:
organisation_id "HEXTORAW(REPLACE(:organisation_id ,'-',''))"

Note: If using a text editor such as NotePad++ the following regular expression can be used for the search and replace:
Find: (.*)\sCHAR\(32\)
Replace: $1 "HEXTORAW\(REPLACE\(:$1 ,'-',''\)\)"

15. This step is only applicable if you have BLOB columns. VARBINARY(MAX) fields in MSSQL are mapped to BLOB columns in Oracle. The migration tool dumps out the hex values in VARBINARY fields as text, oracle then imports this text into a temporary CLOB column which it then runs a stored procedure on to convert it to a BLOB. In the control folder find all the tables that may have BLOB columns and set the char(XX) to an appropriate value depending on how large the data is. Note that the column may have been renamed to SQLDEVELOPER_CLOB_X. See example below.

load data
infile 'data/DEMO.BinaryData.dat'
 "str '<EORD>'"
into table DEMO.BinaryData
fields terminated by '<EOFD>'
trailing nullcols
(
  binaryData_id "HEXTORAW(REPLACE(:binaryData_id ,'-',''))",
  version ,
  SQLDEVELOPER_CLOB_3 char(5564262)
)

Locate the "Orcale_loader.bat" file at the same level as the "control" folder and set the readsize parameter to an appropriate value for each table with a BLOB column.

sqlldr %username%/%password%@%tnsname% control=control\DEMO.BinaryData.ctl readsize=104857600 log=log\DEMO.BinaryData.log

16. Open up a command prompt as administrator. Navigate to the dated directory inside the "datamove" folder.
c:\Windows\System32>cd\
c:\>cd <migration script directory>\<project name>\datamove\<dated folder>

17. Run the following command. If the SQL Server database is hosted on the same box then use localhost for the server name. This command will dump all the data in all the tables into a directory called "data" nested within the above folder structure.

MicrosoftSQLServer_data <server name> <username> <password>

18. Run the following command. The service name will be XE for this scenario. This command requires that the command in step 15 has been run successfully, otherwise you may receive an "lfiopn failed for file (*.log)" indicating that the log folder is missing. A log folder contains details on how many rows were successfully imported and how many failed per table.

oracle_loader <service name> <username> <password>