Pages

Friday, March 20, 2015

JNI / Android NDK

If you visit the Android developer page and read about Android's NDK, it will seam as if Google is trying to scare people from using it and just stick with Java. In most cases this will properly be the best idea. Java is a great language and does a good job for most tasks. However JNI is not as dangerous as Google is trying to make it. Most of Android is build on JNI and IPC. Information is being parsed in and out of JVM and between processes constantly, so why should applications not take advantage of this as well?

One thing to remember about JNI is that it creates a bit of overload to parse in and out of JVM. But depending on the task, this is not necessarily enough downside to keep away from it. The best way to figure out whether or not to use C/C++ or Java, is to test your task in both and then compare the result.

One good example of a task better suited for JNI could be some extensive file operations. For this example we will collect the content of the stat file for all currently running processes on a device. On my device running Android 5, we are talking about rounded 300 files. We add this to a loop which will run 100 times which in turn will create 3000 file read operations.

The first thing we need, is a basic activity to run our examples in.

MainActivity.java:
public class MainActivity extends Activity {
 @Override
 protected void onCreate(Bundle savedInstanceState) {
  super.onCreate(savedInstanceState);
  
  setContentView(R.layout.activity_main);
  
  StringBuilder builder = new StringBuilder();
  
  for (int i=0; i < 2; i++) {
   double start = System.nanoTime();
   int files = 0;
   
   for (int x=0; x < 100; x++) {
    String[] list = null;
    
    switch (i) {
     case 0: list = JavaCollector.collect(); break;
     case 1: list = NativeCollector.collect();
    }
    
    files += list.length;
   }
   
   double end = System.nanoTime();
   
   builder.append(i == 0 ? "JavaCollector" : "NativeCollector\n");
   builder.append("Files = " + files + "\n");
   builder.append("Time = " + ((end - start) * Math.pow(10, -9)) + " seconds\n");
   builder.append("\n");
  }
  
  TextView view = (TextView) findViewById(R.id.textview);
  view.setText(builder.toString());
 }
}

We also need a Java method to do the collection work. We will separate the native method and the Java method into two files in this example. It makes it easier to keep an overview.

JavaCollector.java:
public class JavaCollector {
 
 /*
  * Re-defined regexp to match directories in /proc with process id's
  */
 protected static final Pattern REFEXP_PID = Pattern.compile("^[0-9]+$");
 
 /*
  * The method that collects all the file data
  */
 public static String[] collect() {
  /*
   * Get a listing from the /proc directory
   */
  String[] procListing = new File("/proc").list();
  
  /*
   * Define our input stream
   */
  BufferedReader in = null;
  
  /*
   * Create a temp container for the file output
   */
  ArrayList<String> lines = new ArrayList<String>();
  
  /*
   * Handle each entity in /proc
   */
  for (String procEntity : procListing) {
   /*
    * We only want the pid directories
    */
   if (REFEXP_PID.matcher(procEntity).matches()) {
    try {
     /*
      * Open the sub directory stat file
      */
     in = new BufferedReader(new FileReader("/proc/" + procEntity + "/stat"));
     
     /*
      * Get the content of the stat file
      */
     lines.add(in.readLine());
     
    } catch (IOException e) {} finally {
     if (in != null) {
      try {
       /*
        * Close the file
        */
       in.close();
       in = null;
       
      } catch (IOException e) {}
     }
    }
   }
  }
  
  /*
   * Create and return a string array
   */
  return lines.toArray( new String[ lines.size() ] );
 }
}

And we cannot compare the above Java class to JNI without a native example as well.

NativeCollector.java:
public class NativeCollector {
 
 /*
  * Load our collector library
  */
 static {
  System.loadLibrary("collector");
 }
 
 /*
  * This is really a call to Java_com_example_NativeCollector_collect() in collector.cpp
  */
 public static native String[] collect();
}


collector.cpp:
JNIEXPORT jobjectArray JNICALL Java_com_example_NativeCollector_collect(JNIEnv *env, jobject thisObj) {
 /*
  * Open /proc
  */
 DIR* procDirectory = opendir("/proc");

 if (procDirectory != NULL) {
  /*
   * Create a temp container with minimum 100 indexes pre-allocated
   */
  std::vector<std::string> lines(100);

  /*
   * Create an input stream
   */
  std::ifstream in;

  /*
   * Create a variable for the proc entities
   */
  struct dirent* procEntity;

  /*
   * Handle each entity in /proc
   */
  while ((procEntity = readdir(procDirectory)) != NULL) {
   /*
    * We only want the pid directories
    */
   if (std::regex_match (procEntity->d_name, std::regex("^[0-9]+$") )) {
    /*
     * Open the sub directory stat file
     */
    std::string path = std::string("/proc/") + procEntity->d_name + "/stat";
    in.open( path.c_str() );

    if (in && in.good()) {
     /*
      * Get the content of the stat file
      */
     std::string line;
     std::getline(in, line);

     lines.push_back(line);
    }

    /*
     * Close the file
     */
    if (in) {
     in.close();
    }
   }
  }

  /*
   * Create Java return data
   */
  if (lines.size() > 0) {
   /*
    * Create a Java array
    */
   jobjectArray ret = env->NewObjectArray(lines.size(), env->FindClass("java/lang/String"), NULL);

   for (int i=0; i < lines.size(); i++) {
    /*
     * Place the line in a Java String
     */
    jstring stringObject = env->NewStringUTF( lines[i].c_str() );

    /*
     * Add the Java String to the Java Array
     */
    env->SetObjectArrayElement(ret, i, stringObject);

    /*
     * Release the Java String reference.
     *
     * Note: This is important. We can only have a limited ammount of Java objects
     * at a time and they are not auto released until we return to the JVM. And since we
     * are looping an unknown, but large, number of files, we could end up with a memory overflow.
     */
    env->DeleteLocalRef(stringObject);
   }

   /*
    * Return the array to JVM
    */
   return ret;
  }
 }

 /*
  * Return an empty array
  */
 return env->NewObjectArray(0, env->FindClass("java/lang/String"), NULL);;
}

If we compile and run this application, we will get the following result:
  • JavaCollector$collect: 21.2 seconds
  • NativeCollector$collect: 2.7 seconds

    This is not just slightly faster than Java, this is 87% faster. 
This is only a simple example. If it should be used in a real application, then depending on the amount of files and amount of data, it might be a good idea to make some sort of iteration mechanism to avoid memory overflow. But as the result shows, it is sometimes a good idea to check whether or not a JNI solution might be better. Java is not always the correct solution. 

So get started with:

Saturday, January 17, 2015

Communication across the UI

One of the greater things in newer Android versions is the Fragments. It allows you to split the UI into smaller pieces that can be put together in different ways, depending on different circumstances like screen size and such. However some times you might need to adapt things in one Fragment based on some conditions in another. Talking to an Activity from within a Fragment is easy enough, but Fragments to Fragments is another thing. Especially since it is unadvised to have one specific Fragment talk to another specific one.

A better solution is to have a small message delivery system that works similar to Android's broadcast system. That way you can parse information between Fragments, but without targeting specific ones. For this we need an extended Activity that has these capabilities.

One issue with Java (or not, depending on who you ask) is that Java does not support multi-inheritance. I for one is quite fine with this, but I would like to see something like traits be introduced into the language. In this the problem lies with the many different Activity and Fragment classes that Android has available. We don't want to copy paste everything into each Activity and Fragment sub-class that we want to use.

With the lack of traits or similar, we will be using the famous logic trick instead where we place all the logic into one class, and then only create redirection in each of the Activity and Fragment classes. Not only will this limit the size of each Activity and Fragment class, but it will also make sure that we only have to edit one class if we want to change something.

ActivityLogic.java:
final public class ActivityLogic {
    public static interface IActivityLogic {
        public void onReceiveMessage(String message, Object data, Boolean sticky);
        public void onFragmentAttachment(IActivityLogicFragment fragment);
        public void onFragmentDetachment(IActivityLogicFragment fragment);
        public void sendMessage(String message, Object data);
        public void sendMessage(String message, Object data, Boolean sticky);
    }
    
    public static interface IActivityLogicFragment {
        public void onReceiveMessage(String message, Object data, Boolean sticky);
    }
    
    private final static class ActivityLogic_MessageHandler extends AbstractHandler<ActivityLogic> {
        public ActivityLogic_MessageHandler(ActivityLogic reference) {
            super(reference);
        }

        @Override
        public void handleMessage(Message msg) {
            ActivityLogic logic = getReference();
            
            if (logic != null) {
                IActivityLogic activity = logic.ActivityLogic_mActivity.get();
                
                if (activity != null) {
                    Set<IActivityLogicFragment> fragments = new HashSet<IActivityLogicFragment>(logic.ActivityLogic_mFragments);
                    Object[] input = (Object[]) msg.obj;
                    String message = (String) input[0];
                    Object data = input[1];
                    
                    activity.onReceiveMessage(message, data, false);
                    
                    for (IActivityLogicFragment fragment : fragments) {
                        fragment.onReceiveMessage(message, data, false);
                    }
                }
            }
        }
    }
    
    private Set<IActivityLogicFragment> ActivityLogic_mFragments = Collections.newSetFromMap(new WeakHashMap<IActivityLogicFragment, Boolean>());
    private Map<String, Object> ActivityLogic_mStickyMessages = new HashMap<String, Object>();
    
    private ActivityLogic_MessageHandler ActivityLogic_mMessageHandler;
    private WeakReference<IActivityLogic> ActivityLogic_mActivity;
    
    public ActivityLogic(IActivityLogic activity) {
        ActivityLogic_mActivity = new WeakReference<IActivityLogic>(activity);
        ActivityLogic_mMessageHandler = new ActivityLogic_MessageHandler(this);
    }
    
    public void onFragmentAttachment(IActivityLogicFragment fragment) {
        synchronized (ActivityLogic_mFragments) {
            ActivityLogic_mFragments.add(fragment);
            
            for (String message : ActivityLogic_mStickyMessages.keySet()) {
                fragment.onReceiveMessage(message, ActivityLogic_mStickyMessages.get(message), true);
            }
        }
    }
    
    public void onFragmentDetachment(IActivityLogicFragment fragment) {
        synchronized (ActivityLogic_mFragments) {
            ActivityLogic_mFragments.remove(fragment);
        }
    }
    
    public void sendMessage(String message, Object data) {
        sendMessage(message, data, false);
    }
    
    public void sendMessage(String message, Object data, Boolean sticky) {
        synchronized(ActivityLogic_mFragments) {
            if (sticky) {
                ActivityLogic_mStickyMessages.put(message, data);
            }

            ActivityLogic_mMessageHandler.obtainMessage(0, new Object[]{message, data}).sendToTarget();
        }
    }
}

The above Activity Logic class is only the first step towards getting our message delivery system working. Since this is meant to be used to communicate between Fragments, we still need a Fragment Logic class as well.

FragmentLogic.java:
final public class FragmentLogic {
    
    public static interface IFragmentLogic extends IActivityLogicFragment {
        public IActivityLogic getParent();
        public void sendMessage(String message, Object data);
        public void sendMessage(String message, Object data, Boolean sticky);
    }
    
    private WeakReference<IActivityLogic> FragmentLogic_mActivity;
    private WeakReference<IFragmentLogic> FragmentLogic_mFragment;
    
    public FragmentLogic(IFragmentLogic fragment) {
        FragmentLogic_mFragment = new WeakReference<IFragmentLogic>(fragment);
    }
    
    public void onAttach(IActivityLogic activity) {
        FragmentLogic_mActivity = new WeakReference<IActivityLogic>(activity);

        IFragmentLogic fragment = FragmentLogic_mFragment.get();
        if (fragment != null) {
            activity.onFragmentAttachment(fragment);
        }
    }
    
    public void onDetach() {
        IFragmentLogic fragment = FragmentLogic_mFragment.get();
        IActivityLogic activity = FragmentLogic_mActivity.get();
        
        if (fragment != null && activity != null) {
            activity.onFragmentDetachment(fragment);
        }
        
        FragmentLogic_mActivity.clear();
    }
    
    public IActivityLogic getParent() {        
        return FragmentLogic_mActivity.get();
    }
    
    public void sendMessage(String message, Object data) {
        sendMessage(message, data, false);
    }
    
    public void sendMessage(String message, Object data, Boolean sticky) {
        IActivityLogic activity = getParent();
        
        if (activity != null) {
            activity.sendMessage(message, data, sticky);
        }
    }
}

Now that we have all of the logic behind this delivery system, we need to extend some classes that will be using these classes. Let's start with the Activity class.

AbstractActivity.java:
public abstract class AbstractActivity extends Activity implements IActivityLogic {
    
    private ActivityLogic mLogic;
    
    public AbstractActivity() {
        mLogic = new ActivityLogic(this);
    }
    
    @Override
    public void onFragmentAttachment(IActivityLogicFragment fragment) {
        mLogic.onFragmentAttachment(fragment);
    }
    
    @Override
    public void onFragmentDetachment(IActivityLogicFragment fragment) {
        mLogic.onFragmentDetachment(fragment);
    }
    
    @Override
    public void onReceiveMessage(String message, Object data, Boolean sticky) {}
    
    @Override
    public final void sendMessage(String message, Object data) {
        mLogic.sendMessage(message, data);
    }
    
    @Override
    public final void sendMessage(String message, Object data, Boolean sticky) {
        mLogic.sendMessage(message, data, sticky);
    }
}

And we will of course also need a Fragment class.

AbstractFragment.java:
public abstract class AbstractFragment extends Fragment implements IFragmentLogic {
    
    private FragmentLogic mLogic;
    
    public AbstractFragment() {
        mLogic = new FragmentLogic(this);
    }

    @Override
    public final IActivityLogic getParent() {        
        return mLogic.getParent();
    }
    
    @Override
    public void onAttach(Activity activity) {
        super.onAttach(activity);
        
        mLogic.onAttach((IActivityLogic) activity);
    }
    
    @Override
    public void onDetach() {
        super.onDetach();
        
        mLogic.onDetach();
    }
    
    public void onReceiveMessage(String message, Object data, Boolean sticky) {}
    
    public final void sendMessage(String message, Object data) {
        mLogic.sendMessage(message, data);
    }
    
    public final void sendMessage(String message, Object data, Boolean sticky) {
        mLogic.sendMessage(message, data, sticky);
    }
}

And that's it. We now have one normal Activity and Fragment class that we can use. Note that these are abstract, this is because we use these to extend our actual ones.

Let's take a small example of how to use these.

SomeFragment.java:
public class SomeFragment extends AbstractFragment {
    @Override
    public void onResume() {
        // Send a message to other Fragments or the Activity
        if (someVariable == "someCondition") {
            sendMessage("tellSomeone", "someCondition");
        }
    }

    @Override
    public void onReceiveMessage(String message, Object data, Boolean sticky) {
        // Receive message from other Fragments or the Activity
        if (message.equals("someOtherCondition")) {
            // Do something
        }
    }
}

If we need other types as well, like DialogFragment or ActionBarActivity, we can re-use our logic classes to easily create them.

Friday, October 31, 2014

How does Hacklang work

I must admit that I am not a fan of Facebook. Unlike 90% of the world, I do not, and have no intention of, ever owning an account. The list of reasons are long, and some of the things on that list have more to do with the people using it, rather than the service itself. Something just happens to normal people when entering the website, it's like watching one of the many talk-shows and reality programs on TV.

But although this article is about Facebook, it is not about the service that they are most famous for. This article is about a small extension to the PHP programming language, created by Facebook, called Hack, and the related Virtual Machine called HHVM. If you are a PHP programmer and either do not know about this or if you have not yet tried it, then I'll suggest you have a look at it. This extension provides features that PHP should have had ages ago. This article will not get into what Hack and HHVM is, I'll suggest that you visit the website http://hhvm.com if you are not already familiar with it.

What this article will focus on, is how HHVM/Hack acts. PHP is normally a dynamic typed language, but Hack introduces static typed features, and at the same time keeps compatibility with regular PHP. It's like mixing oil and water, and actually making them mix. The way it works, is that the VM mostly upholds the rules of PHP while the type checker upholds the rules of Hack. In other words, you can without any issues execute code that is not acceptable by the rules of static typing, but you will get errors when parsing it through the optional type checker. The errors from the type checker can also be removed by changing how strict it should be, which can be set pr. file. This allows you to mix dynamic typing and static typing, and still be able to use the type checker on the strict files.

class MyClass {
    public function test1(): void {
        $this->internal( Map{"key"=>1} );
    }

    public function test2(): void {
        $this->internal( Vector{true} );
    }

    private function internal(Map<string, bool> $param): void {
        // Code ...
    }
}

If we execute test1() from a browser, HHVM would not generate any errors. It would simply allow it. If we execute test2(), HHVM would break the request with an error. The reason for this is that HHVM follows PHP rules, which does not provide any generics. PHP however does allow one to define parameter type, which is the only thing that is being monitored for. The type checker on the other hand, would print a lot of errors here, first complaining about the wrong generics types in test1() and then complain about the wrong data type in test2().

class MyClass {
    public function test(): bool {
        return 1;
    }
}

The same applies for the return type. The above example will not conflict with HHVM. The only way to catch this problem, is by the type checker.

So the way that Hack is able to work along side PHP, is that the VM does not enforce any Hack rules. If you want to check your code for type safety, you will have to check it using the type checker. For the most part this will suffice. But the type checker is still a work in progress, and it does contain a few bugs.

class MyClass {
    public function test1(): void {
        $this->internal(Map{"key"=>new Map(array(
            "key"=>1
        ))});
    }

    public function test2(): void {
        $this->internal(Map{"key"=>Map{
            "key"=>1
        }});
    }

    private function internal(Map<string, Map<string, string>> $param): void {
        // Code ...
    }
}

Both test1() and test2() will parse the exact same value to internal(). However the type checker will only report an error on test2(). The wrong generics type in test1() will not be caught.