Rank: 1294 based on 49 downloads (last 30 days) and 3 files submitted
photo

Joshua Dillon

E-mail
Company/University
Georgia Institute of Technology
Lat/Long
33.77701950073242, -84.39553833007812

Personal Profile:

http://www.almostsure.com/

Professional Interests:
statistical machine learning

 

Watch this Author's files

 

Files Posted by Joshua View all
  • 1 - 3 of
  • 3
Updated   File Tags Downloads
(last 30 days) 		Comments Rating
10 Aug 2011 Screenshot semaphore This mex file provides an interface with the POSIX semaphore functionality. Author: Joshua Dillon distributed computing, hpc, interprocess communic..., ipc, large data, memory 16 0
  • 5.0
5.0 | 1 rating
08 Jun 2011 Screenshot sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon distributed computing, large data, shared memory, multicore computing, ipc, parallel computing 29 43
  • 5.0
5.0 | 8 ratings
24 Jan 2011 Matlab Daemon A Bash script & Mex program which allows for #!-style scripting against a background Matlab session. Author: Joshua Dillon unix, linux, bash, daemon, shebang, script 4 2
Comments and Ratings by Joshua View all
  • 1 - 5 of
  • 31
Updated File Comments Rating
01 Nov 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Email me. jvdillon at gmail
31 Oct 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Hi Oscar. Sorry I do not. I don't even have access to a windows machine so it is unlikely that I will be able to obtain this information even in the foreseeable future. If you figure it out though, please do share it with us! :)
19 Jul 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Hey Dan--thanks for the feedback and letting us know about this trick--this is very useful! (I believe this is also confirmed by Przemyslaw's observations.)

On the next update I will add this example to the comments.

Thanks again for the feedback Dan and everyone else. Hopefully we can keep improving shared memory support!

Speaking of which--to any willing to take on the task... There should be a way to make sharedmatrix work better by having it follow and update the reference-to list in the mxArray data structure. This would make sharedmatrix variables behave more like "regular" variables. This is probably the currently most useful feature enhancement. Please email me if you're interested in helping!
05 Jul 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Hi Przemyslaw. You are exactly correct in your observation and I'm surprised someone didn't complain about this sooner! :) In fact, this issue affected me when I was working with a 16gb cell array.

I designed it this way because I *somehow* didn't think of this simple and elegant solution! (I think because I was so focused on avoiding disk usage.) What I will do for simplicity and generality is have a killclone directive which dumps the data to disk, frees it, and loads it into shm.

I will play with alloc idea too. I avoided this orginally because I was concerned that there are too many matlab internals to support--however I could at lease provide a two-dimensional double container.

I will commit these changes as soon as I have a free moment! (Which could be a >month as I am trying to graduate.)

I do want to note that for the meantime, the current program is still acceptable. Letting the system do the swapping is effectively the same, just that it will involve one extra read from disk that isn't necessary.

I was able to load a 16gb cell into a 24gb machine (sharedmatrix would need 32gb) by letting it thrash for 20min or so. I then left the matrix in memory for the next two weeks!.
05 Jun 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon

Hi Guy. As far as either of us could tell, Windows does not natively have shared memory. As I am not a Windows user though, I could be wrong.

I had hoped that we could design a platform independent version, however even within Boost the shared memory API is Windows specific.
Comments and Ratings on Joshua's Files View all
  • 1 - 5 of
  • 49
Updated File Comment by Comments Rating
14 Jan 2012 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon Akbas, Emre
14 Jan 2012 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon Akbas, Emre

Detach doesn't seem to work for this variable:

x = cell(2,1);
x{2} = [1];

A related question: I see that you deepcopy an mxArray to the shared memory by going through all the elements of the array. Is it not possible to copy the mxArray as a whole block using calls like memcpy and mxDuplicateArray?

14 Jan 2012 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon Akbas, Emre

I'm getting "Unable to detach shared memory." errors. What are potential reasons for this?

01 Nov 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon Dillon, Joshua

Email me. jvdillon at gmail
01 Nov 2011 sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon escarcega, oscar

So Joshua! Do u have Andrew´s email?
Top Tags Applied by Joshua
  • 1 - 5 of
  • 21
distributed computing, hpc, interprocess communication, ipc, large data
Files Tagged by Joshua View all
  • 1 - 3 of
  • 3
Updated   File Tags Downloads
(last 30 days) 		Comments Rating
10 Aug 2011 Screenshot semaphore This mex file provides an interface with the POSIX semaphore functionality. Author: Joshua Dillon distributed computing, hpc, interprocess communic..., ipc, large data, memory 16 0
  • 5.0
5.0 | 1 rating
08 Jun 2011 Screenshot sharedmatrix SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O). Author: Joshua Dillon distributed computing, large data, shared memory, multicore computing, ipc, parallel computing 29 43
  • 5.0
5.0 | 8 ratings
24 Jan 2011 Matlab Daemon A Bash script & Mex program which allows for #!-style scripting against a background Matlab session. Author: Joshua Dillon unix, linux, bash, daemon, shebang, script 4 2

Contact us at files@mathworks.com