PhD thesis

I defended my thesis in AI segmentation on 3D image data nearly 12 years ago. The thesis explores techniques to make a computer understand images of the heart and its vessels in 3D MRI and CT.  These were the days before deep learning, and the use of hand-crafted features to detect structures of interest in 3D was quite common. I must say I did not feel content with the techniques I had developed, as I knew all along the way I was using hand-crafted features based on observations that may not necessarily generalise to all different shapes and sizes. 

I went into areas in my domain during my postdoctoral years, and they were spent working at british image research centres where I explored different problems such as representing 3D objects of interest as 2D flat maps, and also tissue classification using machine learning. 

But I came back to my original PhD problem, in later years of my postdoc, and together with a very bright colleague, I was finally able to solve it using a deep neural net. I was somewhat content by then, as I was finally not hard-coding features or hand-crafting them, but the neural net was able to generalise by 'learning' features from training images. 

 In these years, I spent a lot of time doing two things mostly:  

• Innovating, as my post was publicly and charity funded for making progress to improve our understanding of cardiac diseases using large-scale image analysis 
• Giving back to the research AI community, by collecting images, curating them and sharing images with the community to train their AI models. 

 Here is a link to my PhD thesis

Intensity ridges

For the past two weeks I have been reading a lot. Starting off from the Manniseng PhD thesis to Frangi's vesselness filter to cubic B-spline sub-voxel interpolation, I have covered topics mostly that has got to do with measuring vesselness. Right now I am looking at three different approaches for automatically identifying the pulmonary veins of the left atrium:

1) A fuzzy approach where arteries and veins can be separated using local voxel affinity and range of other factors. The technique computes a range of affinities between every pair of voxel in the image. This is a combinatorilly explosive problem, but can be optimized using efficient data structures.

2) A level-set based approach: Arteries and veins can be "presumably" separated by labelling voxles according to the time-of-arrival of two evolving fronts which start off from a point in the artery and a point in the vein. Level-sets are a pain to implement as they must be solved via partial differential equations.

3) Computing intensity ridges. I was reading the paper by Aylward et. al. and it seems like it can be very useful in atleast detecting the starting points of the pulmonary veins (from the left atrium). The ridgeness function is loaded with Frangi's vesselness function which makes it detect tubular like objects such as vessels. The ridgeness should also be computed for every candidate voxel at an appropriate scale. I have also spent a considerable amount of time reading scale-spaces. A scale-space is the domain of images produced by convolving the image with a Gaussian filter at different variances (width). This 'optimal' scale is selected by choosing the variance (of the Gaussian) which gives the maximum response to Frangi's vesselness function.

I have also been implementing the intensity ridge functions. These include Hessian matrix comptutations, Eigen-space analysis of the Hessian and calculation of the ridgeness function. Although there were a few glitches in the program, these have been fixed. Finally, my program is yielding some ridge points (1800 in total in an image). I would soon be looking at how i can go about traversing these ridge points and also applying testing the ridgeness function at appropriate scales.

Skeletonization of vessels

I have not been the most efficient person for the past two weeks. I think I have been thinking too much about other things. I am finishing up my MIUA paper final revision which was accepted for an oral presentation for the upcoming MIUA conference .

For the past week, amongst other things (setting up cricket matches), I have been looking at various vessel analysis techniques especially R. Manniesing's PhD thesis titled "Image analysis in CT Angiography". Here I was stomped by some unfamiliar techniques such as diffusion filters and level set methods.

Diffusion filters are governed by diffusion equations which basically control the amount of diffusion when applying the Gaussian filter to vessel boundaries. The main idea is to come up with a non-linear diffusion co-efficient (a.k.a conductivity coeff.) which stops diffusion at the vessel boundaries thereby preserving the vessel topology. I was interested in exploring this area since I was a little concerned about the amount of noise in some of my new MRI datasets. The application of the Gaussian blur to the images with low variance has actually not done much harm to the pulmonary veins and its branches. Here are some images to prove that in atleast one of the datasets:

And their surface reconstructions show how easy it is to work with the Gaussian blurred image (bottom image):


So I was quite convinced that the Gaussian blur was rather a blessing than a curse. I thought perhaps I should apply it to my previous MRI studies (which are not as noisy as the new sets) and improve the segmentation results.

I diverted my attention by the middle of the week, away from noise-removal to vessel-axis tracking. I felt that this was prime to two things: 1) Eventually enabling the separation of the pulmonary vein from the artery. 2) Eventually being able to determine the number of pulmonary veins to an atrium. Manniesing's PhD thesis gave a very interesting insight into how Level set methods along with prior vessel topology information can be used to segment the vessel-axis. I have been off-beat with Level set methods and spent the remaining week listening to Level set methods lecture from MIT's opencourseware lecture series for their Mathematical Methods for Engineer's course run by R. Strang. A link to the entire series can be found here.

After meditating on a half-an-hour tube ride, I am a little convinced that given that the vessel-axis of the pulmonary vein and the artery can be tracked, they can be separated trivially. However, what worries me most is that it won't be as trivial to obtain the vessel axes at the vein-artery junctions.

off note, I feel compelled to say that I have started taking some interest in world history.

Morphological Operators

I have taken a different approach to the pulmonary vein ostium search problem. 'Primitive' machine vision techniques such as opening and closing can be applied to certain binary images in order to decompose them into their meaningful parts and separate them from their extraneous parts. These happen to be very useful operators. It is derived from mathematical morphology which is a theoretical model for digital images built using lattice theory and topology. So it is back to the basics of machine vision - and I am a little more than glad that I am doing hands-on morphology. I take it as one of the 'must-know' concepts in machine vision.

I will be looking to separate the atrium from its pulmonary veins using these operators. Although surely the operators, on its own, will not be sufficient to do such a task. At this end, I am also looking to possibly implement knowledge-based operators. These operators will incorporate knowledge about the subdivisions (such as their radii, etc.)

Although reading about the fundamentals of mathematical morphology was not great fun, however, I found this book by Haralick to be really helpful. I also found these lecture notes from the U. of Edinburgh informatics site easy to read and understand.

This Friday I picked up more atrium data from my clinician. We are looking at the possibility of exploring a different technique to the atrium segmentation problem.

Removal of Partial Volume Effected Voxels

As I had pointed out earlier in previous posts, perhaps the one and only reason the segmentation algorithm leaks into surrounding structures especially the aorta and the pulmonary artery is because of partial volume effected voxels. We have observed that the segmentation at times will leak into the pulmonary artery through a pulmonary vein. Although, it is physically possible, atleast in theory, for the pulmonary vein to touch the pulmonary artery. However, our images are Angiographic scans where the blood was Gadolinium-enhanced and the only thing that we see after subtracting the pre-Angio from the post-Angio is the blood pool. So, we can rightfully state that the blood pools of the pulmonary artery and the pulmonary vein cannot physically touch and this is definitely caused by partial volume effected voxels. Partial volume effected voxels are voxels that image two or more tissue types, thus giving an intensity level that is a weighted average of the intensity levels of each tissue type.

So here arises a dire requirement to identify partial volume effected voxels and remove them as much as possible. I have been looking at some papers recently such as Ballester et. al., and my task for this week will mainly be literature review of partial volume effect removal techniques.

About subdivisions

A lot of the atrium segmentation computations depend on how the MRI image is subdivided into subdivisions (or basic components) and also partly on the location and spatial distribution of the local maximum and saddle points. Subdivisions are the collection of voxels each of which lead to a local maximum by following a path of increasing EDT values. Now, although intuitively these subdivisions could be thought of as one group of neighboring voxels, however, as I just found out this is not the case. The subdivisions can actually be a disjoint group of voxels as shown below in the figure. Notice that this is one single subdivision that is centered by the local maximum voxel marked in green (by the red arrow). The trail of points indicate the path that leads to the local maximum for a point in the smaller disjoint subdivision.

Here is another image of the same subdivision with the opacity changed to show the opaqued surface.

This is the main reason behind why the boundaries between certain subdivisions dont have a saddle point. For example if we look at the smaller disjoint group in the subdivision shown above in the figure, this smaller group may not have a saddle point with its neighboring subdivisions, since the other larger disjoint subdivision have satisfied the saddle point requirement by having a saddle point with a neighboring subdivision. Ideally we wouldn't have wanted subdivisions to be disjointed in this manner. As explained, this causes some subdivison boundaries to not have saddle points. This has implications in the work we are currently doing where we are trying to locate ostium centers automatically.

Doubts over saddle point locations

I set off to another glorified week when I intend to investigate the possibility implementing an automatic pulmonary vein detector. From the segmented atrium I wish to locate the pulmonary vein drainage ostium automatically so that their diameters can be calculated by some interactive means.

I had doubts over whether the saddle point algorithm was correctly detemrining the points on the image. A thorough close-examination of the algorithm and after some gruesome hours of analyzing the basic component maps I have come to the conclusion that the saddle point algorithm is working fine. I may not be color coding the BC maps correctly at times, so when the saddle points are overlaid over a color-coded BC map, they appear to be located over points which are not boundaries between two adjacent components. This is the case since the color mapper needs to be fine-tuned and by default neighboring components sometime get the same color.

The tube ride today was very instrumental allowing me to finally think of a way the automatic PV drainage detection can be done. I have noticed that the atrium is nicely subidivided at the ostiums for most of the cases. The only thing that needs to be done is to characterize these ostiums interms of the subdivisions or perhaps saddle point diameters.