Sunday, 13 January 2019

Sending an SMS via Intent in Android

Alchemy allows users to donate to their chose charities via text message, which it used to do automatically using Android's SmsManager functionality. Unfortunately, Google have recently updated the Play Store's terms and conditions to restrict SMS_READ and SMS_SEND to applications whose primary purpose is to act as an alternative SMS application on Android (replacing the usual Messages app for example).

I applied for an excemption for Alchemy, but this wasn't granted. This is understandable I think - Android security has a lot of issues, and excess permissions is definitely one of them - Alchemy is a good citizen and only interacts with a single SMS number, but a malicious applicaiton could easily use the same permissions for nefarious purposes.

Without an exemption, there was no choice but to update Alchemy to avoid using SmsManager. This meant instead requesting an existing SMS app to send the donation SMS instead - via an intent. It took me some time to find out exactly how to do this, but the code required to pre-populate an SMS for a user to send is


Uri uri = Uri.parse("smsto:" + charity.getNumber());
Intent intent = new Intent(Intent.ACTION_SENDTO, uri);
intent.setData(uri);
intent.putExtra("address", charity.getNumber());
intent.putExtra("sms_body", keyword);
intent.putExtra("exit_on_sent", true);
if (intent.resolveActivity(getPackageManager()) != null) {
    startActivityForResult(intent, 1);
    donationViewModel.recordDonation(this.donations, charity.getName(), smsKeywordToDonation(charity.getCost(keyword)));
} else {
    Toast.makeText(this, "No SMS provider found", Toast.LENGTH_SHORT).show();
}

The full code can be found on Github. Note that, unfortunately, SMS applications generally do not seem to respect the "exit on sent" request, so the user must navigate back manually. A bonus for this change is that Alchemy now doesn't require any additional permissions from the user. Previous donations must now be stored in Alchemy itself, instead of using SMS history. This may cause some loss of data in the migration, but should result in more robust behaviour from now on.

Sunday, 4 March 2018

Faster python for data science and scientific computing



Scientific computing and HPC developers will probably be familiar with  Intel's C/C++ compiler suite, which can be used to compile your C, C++ and Fortran code instead of the free GCC compilers and can often result in significant performance improvements without changing a single line. Further improvements can be made by swapping out (generally fantastic) open source C maths libraries such as ATLAS or BLAS for equivalent functionality in Intels MKL (Math Kernal Language). Again - this is usually simply a matter of compiling your existing code against Intel's library and can result in very impressive speed gains for very little work.

What has this to do with Python? Most of Python's most famous data science and scientific computing libraries are written in C/C++, with a simple wrapper allowing them to be called easily from python. If you've ever wondered why Numpy, SciPy, scikit-learn and pandas are so much faster than trying to write the same code yourself in native Python, it's because all of the work in a function like np.multiply() is actually carried out in C "under the hood".

Previously, if you had a licence for Intel's  compiler suite you could compile these python libraries yourself and take advantage of Intel's speed boost in your python applications, but this required both familiarly with C code compilation, as well as an expensive licence. However Intel have now made available a free pre-compiled Python distribution with all the major packages (numpy, scipy, pandas etc.) based on the popular Anaconda distribution.  According to kdnuggets Intel have also re-written some common functions entirely for further optimization - in particular it looks like numpy and scipy's FFT (Fast Fourier Transform) functions have been enhanced significantly. Depending on your workload, using this distribution could boost the execution speed of these libraries by 10-50% without the need for any code change.

If you're interested in optimizing Python code that you wrote yourself and isn't available in any existing (C-implemented) library check out Cython as a way of implementing the most performance sensitive parts of your code in C. Unlike using the Intel distribution linked above, converting part of your code to use Cython can take some development work, however even when using the free GCC compilers you'll see a significant increase in speed over native python code.

Monday, 5 February 2018

Pip unable to download packages running in a Ubuntu docker image on kubernetes

I recently ran into a problem where pip was unable to download packages while running in a docker image on a kubernetes pod. The issue seemed to be that it could not find the actual repo to download from - likely due to some kind of networking issue within either docker or kubernetes. The solution turned out to be to create a file at /etc/docker/daemon.json and enter google's DNS servers as follows:

{ "dns": ["8.8.8.8", "8.8.4.4"] }

I was working from a Ubuntu base image, so I created the file as above before installing and starting docker. Keep in mind that as docker images usually don't contain systemmd, it's not all that easy to restart docker once you have installed it, so creating the configuration first is pretty useful. You can find more information on this at https://development.robinwinslow.uk/2016/06/23/fix-docker-networking-dns/

Wednesday, 21 June 2017

Alchemy app released (in beta)

I've finally published the beta version of Alchemy on the play store. It's still in beta, but it seems to be working well. Give it a shot if you're looking for a different way to donate to charity!

Tuesday, 20 June 2017

Memory leak with android linearlayout and Picasso

I recently ran into an issue using Picasso with Android's linearlayout. I was using Picasso to load images from a url into a linearlayout, handling errors with Picasso's build object as follows:

Picasso.Builder builder = new Picasso.Builder(this.mContext);
builder.listener(new Picasso.Listener()
{
    @Override
    public void onImageLoadFailed(Picasso picasso, Uri uri, Exception exception)
    {
      picasso.load(mThumbIds[0]).into(imageView); // on failure just load the default image
    }
});
Picasso picasso= builder.build();


I was recycling the images correctly with convertView (using a viewholder class), but I couldn't track down the source of a memory leak which occured every time a new image was loaded - eventually causing the app to crash. The leak however when away when I stopped using Picasso's builder, instead just using a simple try catch setup.

try{
    Picasso.with(mContext).load(logo_url).placeholder(R.drawable.alchemy).into(holder.imageView);
}
catch (Throwable e){
     Picasso.with(mContext).load(R.drawable.alchemy).into(holder.imageView);
}

Sunday, 28 May 2017

Installing OpenCV3 for Python 3 on Windows

I need to use python for face recognition as part of the server for a charity app I'm working on. The app is supposed to show charity logos, but in the case of smaller charities, sometimes the google custom search I use to search for the logos returns instead a picture with a person in it. I need to identify cases where this happens and use a placeholder image instead.

My first attempt to install opencv was through pip from anaconda:

pip install opencv-python

However this gave me the following error when I tried to import cv2:

ImportError: DLL load failed: The specified module could not be found.

I uninstalled the anaconda version and found a 64 bit wheel of opencv for python 3.6 at http://www.lfd.uci.edu/~gohlke/pythonlibs/#opencv. Downloading this and installing it with pip worked perfectly. I don't know why the original error occured, but it looks like python 3 support might not be great with opencv at the moment.

Thursday, 18 May 2017

Using Python 3 to extract files from an encrypted archive with a password

Recently I needed to use python to extract the contents of a password-protected zip archive and I came across a few issues I thought would be good to document.

Python has a build in zipfile library that is really good at handling zip files, but unfortunately has a few limitations when it comes to encrypted zip files. This is how Python3 can be used to extract a file from an encrypted zip archive:


The first issue that I came across was some unclear documentation for the "open" method of zipfile in Python 3. The open method uses the "pwd" argument to pass the password for the file, but in Python 3 you need to convert this to bytes before calling open. Unfortunately, the library only seems to support CRC-32 based encryption  - meaning that the default linux zip encryption will work, but AES will not. I was also unable to get this to work with 7zip and WinZip.

Sending an SMS via Intent in Android

Alchemy allows users to donate to their chose charities via text message, which it used to do automatically using Android's SmsManager ...