I, Curtis

Category: Programming

  • ExIf 35 – Re-evaluating an Abandoned Project

    I started the ExIf 35 project (http://exif35.pcfire.net) in 2010 when I returned to shooting film. Perhaps I’m metadata obsessed, or just crazy, but I decided I wanted good ExIf data on my scans of film photos. In the six years since I started the project I allowed myself to reach a point where I felt that ExIf 35 was “complete”, it did what it needed to do, usually pretty well. I made no major changes to the software since 2012 (see: http://exif35.pcfire.net/viewnotes.php) and I drifted somewhat away from film photography as a hobby and art again. This spring I purchased a Nikon F6 camera and dove right back in just as deep as I had been before with film photography – the only difference is that the F6 does a bit of its own metadata recording, which makes having a metadata/ExIf workflow even more important. Since the Nikon F6 was released in 2004 many of its accessories (MV-1) are no longer made and its intended software no longer functions in the modern era, so I had to seek out other alternatives and build new things into ExIf 35 to accommodate my needs.  When I speak of “my needs”, I refer of course to the fact that I record information beyond basic ExIf data. I like to record things like photographic filters in use and details about how the film was developed (see: http://exif35.pcfire.net/docs.php?doc=xmpSpec for a complete list of my special fields).

    Yesterday in a fit of boredom between semesters I began looking at some of my data in Exif as it was stored in Flickr. I realized that there was no degree symbol in my develop temperatures (20 C vs. 20° C), so I started tinkering with the data writing process, only to find out that putting that character in one location in the file (Exif:ImageDescription) caused Flickr to corrupt the field when it is displayed. In an effort to correct the problem I began looking at the field in other applications (ACDSee, PhotoMe, etc). I discovered two things 1) There is nothing wrong with the bloody thing and 2) There are a few new ExIf fields that I had not seen before. This led me to the ExIf documentation (http://www.cipa.jp/std/documents/e/DC-008-Translation-2016-E.pdf).

    Many of the fields that were added in 2012 to version 2.3 and in 2016 to version 2.31 are fields related to sensitivity speed and sensitivity type.   Most notably, ISO is no longer the only way to measure sensitivity. While I doubt I will use anything else, this represents a bit of an evolution in the way that metadata capture on imaging devices is represented. The addition of fields such as PhotographicSensitivity, RecommendedExposureIndex and a new ISOSpeed field expand options for the level of detail that is recorded for film sensitivity, although the creators of ExIf intended such fields for describing electronic sensors and their gain ranges – funny how those parallels between digital and film are maintained after all this time. When I first read about those fields my first thought was a very quirky film called Ilford Delta 3200. This film is marketed (recommended, even encouraged in DX) to be ISO 3200.  The actual film speed is ISO 1000. The film is regularly shot anywhere from ISO 800 to ISO 6400 (some claim it can push to 12,800). Once I figure out what each of these fields does in practice (through reading them in a variety of applications) and maybe through reading ISO specification 12232 I will be better prepared to use these fields to accomplish my goals. Some of the new fields relate to details of the camera and lens configuration used – these I am already implementing as they are clearly defined.

    I feel that I need to re-evaluate ExIf 35 because there are a lot of gaps in what it logs to files, what is collected from users and perhaps some difficulties with the way that it writes data in general. I have been away from the project for too long and have only added fields on occasion. The overall functionality of the application needs to be checked for compliance with modern standards as I implemented my ExIf writing in an experimental way, guessing at how things worked and then leaving things in place once they worked, which is perhaps not the best way to approach something as intricate as metadata. There are also fields that I have not tested in detail how they work, which may also introduce problems that cause some applications to not read the file accurately. The addition of new fields such as time zone offset present cases where the UI and the data storage (preferences and defaults) functions need to change to match the expectations of the standards. Adding the new ISO fields I mentioned before will require a change to the handling of recording ISO data, such as I will need new fields in the form to collect the data and will need to establish logic to ensure that all of the data is recorded accurately and consistently even with those fields are not filled out (very often I shoot and develop my standard film, HP5+, at its DX speed, so assuming the other values is fine).

    The project, and its documentation, are long overdue to be refreshed. There are many areas I have patched in new fields or extended functionality without reworking the workflow or the ExIf encoding process.

  • Comma Separated Volumes

    In my professional work I have had to do a lot of custom coding for things, just because people I work with are “special” and generally don’t do things in a standard way. I’m fine with that, actually, I love that. There are some things that I feel as though I shouldn’t have to “custom code” for though, reading from CSV (comma separated values) is one of them.

    With the project I’m currently working on, I need to read data in from a CSV file (plain text file, columns are separated by commas and there is a header row). The file is about 27,000 lines long. Biggest problem is that the columns aren’t always in the same order and sometimes there are different (additional) columns present. I thought for sure that going into the world of open source I would find some sort of pre-written tool, class or thingy to read the file for me and let me get the columns I want from each row. So far, my quest has turned up nothing, since apparently these files are always predictable and you just are supposed to just Split() the lines as they are read in and capture the indexes you want. Seems simple enough except for the fact that I have a file that is somewhat unpredictable (as in one iteration may have 26 columns, the next may be 28).

    My problem is that I am caught slightly off-guard by the fact that there is nothing pre-written. How is it that such a simple thing has not been written? I suppose once I have finishes my coding it will be my responsibility to clean up the class and release it for the masses.

  • GITI v3

    Working on GITI’s third version has been harder than any of the previous versions. V1 was an experiment that grew into a mostly-functional personal information manager. From the start is barely did anything beyond schedule items and some basic educational tracking stuff (assignment-level stuff). V2 grew out of V1 being structurally inadequate for the mission it was to take on. The change over to V2 was filled with struggles, but it wasn’t too bad because all of the V1 modules were mostly compatible with V2 except for some minor encapsulation changes.  V3 presents more of a challenge.

    Whereas for V1 and V2 I was the primary person responsible for writing “The Interface” (the back end that runs and “is” GITI), for V3 Chris has taken on the responsibility of writing that portion of the code, as well as some additional supporting frameworks. Chris has written an Interface that is drastically different than any of the previous versions of GITI and therefore all of the modules are completely incompatible and have to be adjusted or re-written to make them work. On the surface this is a pain and a problem, but in reality, it gives some much needed revitalization to code that in some cases may have not been changed in 10 years.

    Earlier today I began reading the Wikipedia articles for Personal Information Management and Personal Information Manager. Reading those articles reminded me a lot of what GITI is supposed to be. GITI can’t be just a simple piece of software, it has to take on the personality of its users to be useful. The personal information manager article notes that one of the key problems facing PIMs is that they become a part of a fractured data ecosystem, which ultimately is against the goals of personal information management. GITI started off handling address books, schedules, todo lists, assignments, journals and a variety of other information. Since that time it has become more practical to keep address books in a cloud service like Google Contacts, same with schedule on Google Calendar. Journals have been moved to WordPress. The only thing remaining is the data that GITI stores on educational activity (assignments, class registration, etc) and a few other items that aren’t easily tracked through other tools (like footwear utilization). Some aspects of personal information management have passes beyond the scope of GITI’s usefulness and become a ubiquitous part of mobile computing (or ironically “ubiquitous computing” as it is being called recently). Address books and calendars have always shared a dual space between PIM and PDA tasks and now smart phones cross the bridge between the two, so perhaps those entities no longer need a home in GITI beyond a token level of access? I would love to be able to see my external calendars in GITI, but I don’t really have a need to manage all of that information from GITI. At some point trying to do everything in GITI is just reinventing the wheel and does not go anywhere productive. Some modules have outlived their useful lives and need to be retired instead of being brought into the new version, while other modules (new modules) need to be written to reflect the needs of the users.

    While the old modules are being converted by Chris and myself, I am also working on writing a new module to handle domain names. I have had this “need” in a PIM for a while, but it was never really possible to practically implement a module to take care of some of this stuff until last year when I gained access to my registrar’s API. This is another situation where I really have to focus on the fundamentals and not reinventing what has already been done. The utility provided by my registrar to manage domains is great, but it is nice having access to that data inside GITI as well. I am sort of merging my previous host registry/WHOIS tool with the APIs to create a functional information source to take care of my needs.

    V3 is far from complete and it is questionable whether or not it will be ready for the semester, even as a fully-functional experiment. There is much work to be done, but hopefully the inspiration will strike to put things in the right direction for an orderly transition to the new GITI.

  • ExIf 35 Build 20130520

    ExIf 35 0.2.20130521
May 21, 2013
--------------------

Release Information:
	*Released as Windows Executable
	*Released as Windows Installer

System Requirements (PC):
	*Must be running Microsoft .NET Framework 3.5
	*Supports Windows 2000 and higher
	*Software is developed and tested on Windows 7	
	
Changes:
	*Implemented the ability to import exposure data from the 'EXIF4FILM'
            Android application by CodeUnited. 
	*Added ability to edit data across multiple exposures at the same time
            (Additional Information > Add Information to Multiple)
	
Support:
	*Email EXIF35@PCFIRE.NET
	*Website: http://exif35.pcfire.net
	*SourceForge: http://sf.exif35.pcfire.net

Developed By:
	*Curtis M. Kularski

  • Making ExIf 35 Mobile

    The first application that I intend to write for Android will be a mobile version of my own ExIf 35 application. I have chosen to do this for several reasons. First, I need an application that has a clear purpose to write for my first application while I am learning. Second, at the current time the desktop version of the application relies on my use of a notes application for iOS or Android (carrying the device with me) or carrying a physical notepad and pen. This also requires me manually keeping track of a time stamp, which is in itself a waste of time.

    The first issue is fairly straight-forward. Its harder to write a good piece of software if you don’t know where its going, and you really can’t put a lot of effort into pre-designing the application because you don’t know exactly what the constructs of the development environment entail. Through using ExIf 35 as my base I know what its supposed to look like, what its supposed to do and I have a rough idea of how to get there, but the new environment still takes time to learn. Another advantage of using ExIf 35 is that the object classes are already written, so I don’t have to spend valuable time in the learning process trying to write a new object. My Exposure.cs and Roll.cs will be attached to the new application just as they are to the old one, and the cool thing is, if this works out, then I can keep them synced with each other and not have to ever unlink them (they are objects, thus not platform or environment dependent). While this means that I do still have to write and compile in C#, I am not bothered by this idea. Mono (open-source version of .NET) is a good and usually efficient set of libraries which should work well for what I am trying to accomplish.

    There have been complaints about  ExIf 35 not being mobile since the day I released it publically (literally, same day someone on Flickr demanded an iOS variant). I have to agree with these complaints, since I still have to use a notepad app on my phone or on my iPod to keep track of my exposures until I get back to my desktop to transfer them. This is a waste of time as it requires me entering the data twice. I considered writing a method that could use a standardize text file as input, but unfortunately humans aren’t very good at doing things in a standard way and we use so many personal conventions in recording things that it would require as much time to train a human to write the file as it would to write the application to do it automatically. The other issue is time-keeping. It is important for the purposes of ExIf  35 that times be kept fairly well and as such it is inefficient to have a human keep time, it just happens. I have also found that in using a notes application or a physical notepad I am less likely to enter the extended details of an image (filters used, whether it was close, macro or far, or any number of other settings) than if I were entering it in the desktop application immediately after shooting. While I do not anticipate having all of this information recorded from the start of the mobile app, it will be simple to use since the object class will be maintained.

    The hard part for me about writing the existing application over will be trying to remember how I did it. At present the application opens to the last file known, it knows to create a new file if there isn’t one and any number of other little goodies that I don’t even remember because it comes together so seamlessly. I don’t remember exactly what functions I wrote first before, and thus I will have to start from scratch at remembering how to do them again. A lot of code will be borrowed from ExIf 35, but transferring such a complex set of code over to the new application will result in a number of problems until I get  everything sorted out and linked with where it is supposed to go, so I won’t do very much of that until the basics are implemented.

    A final concern I have about writing the new application is its impact on its desktop predecessor. I will not immediately have the capability to sync data between ExIf 35 PC and the Android version, so I will mostly work on an active file and then transfer it back to the desktop at the same time as I take the roll out of the camera and only use the desktop version for making changes or for writing the data to the image files. This changes the primary role of the application and therefore I will have to re-evaluate the design of the desktop version. Its going to be an iterative process from borrowing the code through the impact on its predecessor.

  • Why I will never write for iOS

    Mobile application development is becoming a big deal in computer programming lately. I am very late to this particular game.  I have considered writing mobile software since the initial releases of ExIf35, when people asked for a version that would work on iPhone. Unfortunately, I have decided that I will never write for iOS as long as Apple maintains its present developer-hostile atmosphere to application development.

    Here is a short list of reasons why I will never write for iOS:

    • Fees: It costs $99 per year to write for iOS. This is not a lot of money for a developer who will profit from their applications, but it is a deterrent for a hobbyist or anyone who doesn’t plan to profit from their software. It can be a fun activity, it doesn’t have to become a profit-seeking enterprise.
    • Platform dependence: While it is technically possible to write for iOS on any platform, Apple has certainly made it simpler to write for iOS on their own OS.
    • Gatekeepers: Apple must verify and certify every application in the App Store. While this may seem like a good idea for the protection of the end users, it hinders the type of applications that can exist in the marketplace. Anything that competes with their own offerings are quashed and anything that may be for an adult audience runs the risk of not being approved at all. Also, while Apple will defend their own apps from being duplicated, there are many applications that seem almost like duplicates in the marketplace. Try looking for a flashlight app for iOS, there are several hundred of them.
    • Hardware Limitations: iOS applications can run only on iOS, same can be said for all platforms (kind of a duh I guess), but iOS only runs on Apple hardware, narrowing the scope of what can be done. Other platforms that are more open run on a variety of hardware, while this in itself can be a challenge, it opens up the possibilities for what can exist.

     

    At this time I am planning to begin trying to write mobile applications over the summer, at least in a limited capacity and as a learning experience. I will be developing for Android only. I have more iOS devices than Android devices, but I think that ultimately the Android development will be more fruitful and useful.

  • ExIf 35 Build 20121117

    I spent some time working on ExIf 35 today. I got a large amount done regarding the implementation of auto-complete fields in both the main interface as well as in the add exposure window. This is only the third new build that I have produced since I stopped production of new builds in June 2010. I am slowly working back into working on ExIf35. I have lost sight of my goals with the application, but am gradually relearning how to use it for my own purposes. My work from today is reflected in my release notes file, copied below.

    The build can be downloaded from: http://exif35.pcfire.net/downloads.php
    The updated XMP specification: http://exif35.pcfire.net/docs.php?doc=xmpSpec
    The updated fields list: http://exif35.pcfire.net/docs.php?doc=fields

     

    ExIf 35 0.2.20121117
November 17, 2012
--------------------

Release Information:
	*Released as Windows Executable
	*Released as Windows Installer

System Requirements (PC):
	*Must be running Microsoft .NET Framework 3.5
	*Supports Windows 2000 and higher
	*Software is developed and tested on Windows 7
	
	
Changes:
	*Change in functionality for roll number
	*New roll number now calculated as maximum observed + 1
	*Development temperature added to Additional Roll Information
	*Autocomplete added to: aperture, shutter speed, focal length
	*Autocomplete added to: ISO
	*Adaptive autocomplete added to: Auxilary Lens/Filter
	*Adaptive autocomplete added to: Developer Solution, Film Type

Notes:
	*Development temperature is accepted in C or F, but will be stored
		and recorded to ExIf as C. Appropriate conversion will be made.

	*This build updates the XMP specification file and the fields
		documentation. 
	
Support:
	*Email EXIF35@PCFIRE.NET
	*Website: http://exif35.pcfire.net
	*SourceForge: http://sf.exif35.pcfire.net

Developed By:
	*Curtis M. Kularski

  • Reconsidering the ExIf35 Panel

    image

    Is this box intimidating? After being away from it for almost 2 years it was intimidating to me when I returned to it a few weeks ago.

    Obviously this is not a useful form for anyone that is not both an expert at photography and in data entry. I think this form is absolutely horrendous for basic day-to-day data storage. At the present time I am considering replacing this particular panel with a more slim model and placing this form as a special option for experts and for use on a case-by-case basis where more information may be needed.

    I believe the revised form shall only need a few pieces of information:

    • Exposure Program (A, S(T), P, M)
    • Aperture
    • Exposure Time
    • Focal Length
    • Time of Exposure (Date Time)
    • Exposure Bias

    These options would reflect the basic pieces of data that are captured and stored by DSC ExIf at exposure time. While the other data, such as metering mode may be useful it is not something that the average photographer changes very often. Orientation is no longer needed because the user can now adjust that after the images have been mated with their data and visually confirm the transform.

    The area that bothers me about making these changes is the loss of special fields such as lens selection, lighting and flash info just because of a need to make the form more user-friendly. I am considering the idea of a pop-out panel that will allow the user to select these pieces of data.

    Another area that is particularly difficult for me at the moment is considering what my data-retention policy in the application will be. I can usually safely remember user and lens. What about the other form elements? Should they keep their values or reset to zero when they are hidden?

    There is a lot to consider regarding the improvement of the user experience for ExIf35.

  • ExIf Rotation Implemented

    image

    ExIf 35’s rotation functionality now completely works!

  • Adding Rotation to ExIf35

    image

    About 2 years ago when I thought I wanted to do film photography I started working on an application called ExIf35. The idea was to have the same level of data storage in JPEGs from a film scanner as those found in DSC (Digital Still Camera) image files. One of the biggest pains that I had was the confusing way in which ExIf rotation is handled. The old method required entering the rotation information when you first entered the data for the exposure. While entering aperture and shutter time is hard enough, the application expected the user to remember which way the camera was turned as well. This was of course a step up from the user needing to enter the actual ExIf integer for which way the frame was to rotate.

    Now I am working toward developing the form pictured above, it includes the same drag/drop or select tool that is used in the original ExIf writing process, except not requiring an output directory (because this form just saves to the data file, not the actual image file). Once all files are recorded the user can click on an image file to get a thumbnail image. Once I get the code written the blue and green buttons can be used to change the direction of the thumbnail and record the proper alignment for the ExIf notation. The data will then be saved to the ExIf35 exposure record in the roll file and the information can be written as normal.