Tuesday, December 14, 2010

Eclipse: Great or Overrated?

I've been working on integrating Openfire and vBulletin, to allow a vBulletin based community to have a full fledged instant messaging service. It's been challenging at times, but mostly fairly easy. I'd already done some work learning how to authenticate with a vBulletin community. My biggest problem has been, surprisingly, Eclipse.

Wednesday, November 24, 2010

Making an MP4 Video

I've been working on switching any/all of my older videos from their original format (.mov, .avi, etc) into an .mp4 container. This will let my PS3 stream the videos, allow my MythTV setup to stream them, and even allow my phone to do the same. It's pretty nice, actually.

Tuesday, September 14, 2010

Announcing gogo, The Bash Project Switcher

If you're a developer, then you've faced this problem: When you switch between multiple projects, you have to switch your shell, your editor, and possibly a lot more. If it's time to start a new project, then there's all the busy work of making a new project.

Those problems are now ending.

Monday, August 9, 2010

Steal This Idea: Writer's Tools for Web and Android

Okay, yes, I have an Android phone, and I like it quite a bit. This idea would allow a story author to maintain notes on the phone/tablet and synchronize them with a web site. More than that, though, it would include features to help writers keep their ideas organized. A good example of what I have in mind is something like the Writer's Cafe, but written for the web and Android, instead of being only a local app. Even though Writer's Cafe is very good (and it really is) being only a local app does make it harder to use than I would want it to be, personally. It also doesn't support all of the things I would want to see supported, which (in particular) includes the support of audio and video notes/clips. Using Android and Web should make these items considerably easier.

Friday, August 6, 2010

Python, Soap, and Stub Libraries

Last week, at work, I came across a new problem for me. Specifically, I was working out how to submit information to a set of SOAP servers from a Python program. This was supposed to be the easiest part of the whole project. It turned out to be the hardest. Why?

Monday, August 2, 2010

Steal This Idea: Smart Phone Becomes Universal Remote Control

This one will take a bit more work than the other ideas, mainly because this one will require some hardware work, too. I want to see my Android based phone able to act as a remote control for all of the devices in my entertainment system. This means Bluetooth, UPnP, DLNA, and IR. The first three are pretty easy, but the last will require the creation of some sort of IR transmitter than can accept commands from my phone, and turn them into IR commands to control my other devices. It should, of course, be fully programmable, learning, etc. Noticeably harder to accomplish, but I think well worth the effort.

Friday, July 30, 2010

Steal This Idea: Electronic Emergency Preparedness Kit

When Hurricane Katrina hit New Orleans, I remember reading somewhere about a person who recommended a full electronic version of important documents, stashed them on a USB key, and stashed the USB key into his emergency kit. I'm looking at my Android based smart phone and saying "Wait, why not use this?" I'd want any program to have top-notch encryption and allow me to store any images I choose. This would allow me to store mine (and my family members') driver licenses, marriage certificate, home ownership papers, birth certificate, social security cards, everything. Of course, all of these would need to be encrypted. This would help to ensure that, in the event of an emergency where I lose all of those originals, I could use the information on my phone to at least find out the things I need to get them replaced, and restore my life.

Monday, July 26, 2010

Announcing the Availability of tgext.menu v1.0b3

tgext.menu v1.0b3 is available for immediate download. It can be found on PyPI in an easy_install'able form. The source may be found at BitBucket. This release now allows specifying the icon for a menu entry in the decorators and functions, fixes an issue where an id could be duplicated, and allows for replacement of the default template.

Friday, July 23, 2010

Steal This Idea: Shared SmartPhone Shopping List

I've been using some form of PDA type device for a long time now. I used to use a PalmOS based device until last summer, when I switched to Android. On PalmOS, I had HandyShopper, and it was a shopping list done right. I would love to see HandyShopper done for Android, but with one additional tweak: It should have the ability to synchronize with a Google Docs spreadsheet. This would provide a nicely open way for someone to maintain a shared shopping list with someone else on a completely different phone. GroceryIQ comes very close, but it suffers from some speed issues.

Wednesday, July 21, 2010

In Praise of MythTV Developers

I use MythTV, and have done so for a very long time. I think that, right now, we're looking at about 7 1/2 years. I have no intention of giving it up, it's just too useful.

Just two weeks ago, I got my new phone, a Samsung Galaxy S. Beautiful phone. It supports using DLNA to stream video. DLNA, it turns out, is a subset of UPnP. MythTV, it turns out, has had a UPnP server built into it for a long time.

So, in theory, I should be able to stream from MythTV to my phone. The reality wasn't so great. I tried, sproadically, for about a week. Then, on Monday night, I decided to get serious. I did all the groundwork to find out what was going on, and found out that MythTV had a problem.

I submitted a bug report, and included all of the details that I knew, including purpose, what I had done, everything. Tuesday night? By the end of the night, we had a working patch for MythTV that will be included in their next release. That same patch works against the version that ships with Ubuntu.

I can stream from MythTV to my phone without a problem. And the MythTV folks fixed my problem within 24 hours. That's seriously amazing work guys.

Thank you. May you continue to enjoy the success you have so far, and may ever more of it come your way.

Monday, July 19, 2010

Steal This Idea: CarPC with Android and DLNA

Lots of folks have their own Car PC built and doing things already. This, though, is slightly different. Use an Android tablet as the base. Give it some extra storage somehow, so it can hold videos, music, and the like. Hook it up so that it can read OBDII from the car. If the tablet can handle its own internet connection via a SIM card or some such, do so, or else have it use a tethered phone of some sort for on the road. Hook the tablet into environmental controls for the car. Do some hardware hacking with antennas, and make it so that the tablet can positively identify the location of all phones in the car. allowing people to customize their environment as much as possible. Ensure DLNA support, and get DLNA support on all the phones (Android, iPhone, etc) to allow people to share their media wirelessly with the main tablet and with each other.

Friday, July 16, 2010

My Issues With Perl, Redux

I recently had a conversation with a colleague about Perl, and I kind of screwed up my explanation of why Perl drives me nuts. He started talking about Perl's object model, and I tried to explain why Perl doesn't really have one, and things went downhill. I've been thinking about it since then, and my single biggest issue is not with the object model, nor the ugly syntax. No, my issue is with what happens when a given function/method gets called by something.

Thursday, July 15, 2010

TurboGears, SQLAlchemy, and Parent/Child Relations on the Same Model

Best thing to do is to read this post. I'm just posting it here to try to help it get more exposure to those who might need it.


Steal This Idea: White Board

I want a simple whiteboard. The problem with most paint applications of any sort is that they attempt to grow into something truly grandiose. The problem for me is that, sometimes, I need something that is extremely simple, and a whiteboard is about as simple as it gets. Using a real world one, I can pick a marker color, or an eraser. That's it. I want a virtual one that provides me just as simple a set of options. I want it on the web, I want it on the local desktop, and I want it as an Android application. The only add-in features should be an ability to change the background color, maybe some sort of lines (like ruled paper, or grid paper, or hexagons), and some way to save and search for them. Anything more than that, and it becomes way too complicated for rapid fire drawing of an idea, and that is where the whiteboard will always win.

Monday, July 12, 2010

Announcing the Availability of tgext.menu v1.0b2

This is a minor update. It contains new installation instructions due to testing failures right after installation in a new quickstart, along with a fix for a menu display bug with jdMenu, and some new API enhancements from bitbucket user scottawilliams. It is already available on PyPI, and you may upgrade as soon as you are ready.

Steal This Idea: Media Library

I have a lot of different media in my life that I want to track. I have document files of all sorts (word docs, text files, pdf files, epubs, etc), photos, videos, and music. I want to be able to organize them all in one place, tag them all, search them all, and display them all. I want it to be a web interface, and I want to be able to set up my own server hosting it. I want the files to be something that can be read by way of a plugin library, so that I can easily add support for unknown formats in future. I want it to support external sites using it (so that I can tell youtube to pull from it, or tell my ebook reader to pull from it, etc). It should support OPDS for textual information (epub, html, mobi, pdf, doc, etc), and Gallery 2's remote protocol for visual media (videos, photos). Since I'm a Python and TurboGears fan, I'd love it to be written in TurboGears.

Sunday, July 11, 2010

Unit Testing Redux

I've mentioned unit testing before. I've extolled its virtues before. I'm not going to go over those items today. Instead, I'm going to go over how it saved my project at work.

Saturday, July 10, 2010

Steal This Idea: RPG Assistant

This is just the first post of many I will make like this. I've long had in mind an RPG Assistant. It should allow storage of all the information about a given campaign world, including maps, creatures, characters, plot hooks, current events, everything. It should be platform agnostic (using something like wxWidgets, wxPython, Qt, Python-Qt, whatever, and you will maintain easy cross-platform compatibility with Windows, Linux, and Mac). A really nice addition to it would be to have an Android version compatible with the other versions. It should also be game system agnostic, so that any version of D&D can be put in, along with GURPS, TMNT, Rifts, Torg, Fudge, etc. I hope to be able to write it up someday, I really do. But, if not, I'll just be happy seeing it actually happen.

TurboGears and Amazon EC2 Benchmarking, Part 4: Conclusions

Reviewing my methods, and the results I got, I found out quite a bit about TurboGears. I learned about how it will perform in real world scenarios, and I learned how to figure out the level of hardware I will need to accommodate a community. So, what does this mean for you?

Friday, July 9, 2010

TurboGears and Amazon EC2 Benchmarking, Part 3: What I was Testing

I actually learned a few things, some of which were surprising, some of which were not. One thing that I don't think I've communicated very well to my readers is that I most emphatically am not comparing TurboGears to any of the other frameworks out there. I'm comparing TurboGears to itself, and finding out what I need to do to scale it up.

Tuesday, June 15, 2010

FreeBSD 8, Apache 2.2.14, Python 2.6.4, mod_wsgi 3.2 compile failure

This is a keyword laden post. I spent far too long trying to figure this out, and Google could not help me. Turns out it's really simple, but getting there was challenging, to put it mildly. Here goes:

/usr/local/include/pth/pthread.h:537:1: warning: this is the location of the previous definition
mod_wsgi.c: In function 'wsgi_discard_output':
mod_wsgi.c:11599: error: 'apr_bucket_type_t' has no member named 'pth_read'
mod_wsgi.c: In function 'wsgi_execute_remote':
mod_wsgi.c:12112: error: 'apr_bucket_type_t' has no member named 'pth_read'
apxs:Error: Command failed with rc=65536

If you are using the platform above, and get an error message that looks like that when building mod_wsgi 3.2, the problem is due to a configure option in your Python installation. Take the following steps:

cd /usr/ports/lang/python26
make config
make deinstall distclean
make install

Update the configuration, and disable GNU Pthreads. I'd also turn on SEM and THREADS, so that you can get the benefit of the multiprocessing module in Python 2.6.

Now, time to fix up mod_wsgi

cd /usr/ports/www/mod_wsgi3
make deinstall distclean
make install

And that will do it. You should now get everything working cleanly.

Announcing tgext.xmlrpc v0.8

This is an important update, and also one that is 100% backwards compatible.

This update allows you to have nice Python method signatures, as opposed to the hacky "*p, **kw" trick that v0.6 required on all methods.

To clarify, an example. v0.6 code required this:

class MyXmlRpc(XmlRpcController):
    @xmlrpc([['int', 'int', 'int']])
    def sumthem(self, *p):
        return p[0] + p[1]

v0.8 finally corrects this ugliness. This now works:

class MyXmlRpc(XmlRpcController):
    @xmlrpc([['int', 'int', 'int']])
    def sumthem(self, i1, i2):
        return i1+i2

Note that any code written for v0.6 will work just fine with v0.8. This is a purely cosmetic change, though a welcome one, I think.

It is available via PyPI and BitBucket (with documentation also on BitBucket).

Note that I did not forget to finish my Amazon EC2 benchmarking series. The next post in the series has been bugging me. I might not be William Shakespeare, but I do have some standards for my writing, and I've not been able to write up anything decent yet. I think I've finally figured out why I hated what I was writing, so will finish that series this week.

Wednesday, June 9, 2010

TurboGears and Amazon EC2 Benchmarking, Part 2: Requests per Second

That all important question has been asked, and is now answered: How fast is TurboGears? I started this series yesterday when I described my methodology. I conclude soon with my analysis of what I did, and what I found.

Tuesday, June 8, 2010

TurboGears and Amazon EC2 Benchmarking, Part 1: Methodology

This is going to be a multi-part series. As of right now, I've got at least three parts, and I may be adding another one in. I've just gathered a lot of data from Amazon, and the TurboGears EC2 images I made. Enough of it, in fact, that I'm not sure how to present all of it. For now, I'll explain how I gathered the data.

Monday, May 31, 2010

Announcing TurboGears EC2 Images

Amazon has long offered a service whereby the community can purchase time on their clusters. Today, TurboGears is able to take advantage of this in a new way. It is now easier than ever before to get a TurboGears instance up, running, and viewable by you, allowing you to find out how that TurboGears based application really looks and functions.

Monday, May 24, 2010

Announcing tgext.xmlrpc

One of the questions that we in the TurboGears community get asked is "How can I have XMLRPC in my TurboGears application?" Until now, we didn't have a useful answer beyond "Write it up". Now we have tgext.xmlrpc.

tgext.xmlrpc allows you to have an XMLRPC hierarchy. Your code becomes a case of simply setting up your controller and working with it like any other controller. So, if you need to have XMLRPC in your TurboGears application, check it out (and don't forget to read the docs!). I think you'll find it quite helpful.

Wednesday, April 7, 2010

What To Do When It's Time To Find A New Maintainer

This post is going to ramble a bit, and I apologize. Almost two years ago, I got an Asus EEE 901 netbook. This is a great little device, letting me have a highly portable laptop style computer. I then got the idea to use it for GeoCaching, and wrote Cache 901. The problem I face now is that it's time for me to hand Cache 901 over to somebody else.

Friday, March 26, 2010

Emacs As 3Way Diff Tool, Use With Mercurial

So, I've switched recently to EMacs. Mostly doing okay with the switch, but today I wanted to do something I'd not seen mentioned elsewhere: I wanted to use it as a 3 way diff tool, and integrate this with Mercurial. Turns out it's not too hard to do at all.

Friday, March 5, 2010

JSOn vs XML vs YAML, and Python Parsing

I've begun working on translating a board game into a computer game. In order to do so, I need to be able to represent the state of the game and its many (many) tokens. In addition, I need to be able to store and load that state quickly. Finally, I need that state to be something that I can represent with a single string.

Monday, March 1, 2010

Announcing the Availability of tgext.menu v1.0b1

It's been a lot longer in coming than it should have been, maybe, but it's here finally: The menuing extension for TurboGears. This extension gives TurboGears developers the ability to stop looking for where their controllers are set, how to integrate new controllers into their navigation menus, etc.

All of that goes away. Now you can just use simple functions and decorators to generate your menus for you. As it also uses jdMenu, you get dynamic menus that are very suitable for a web application.

If you're a TurboGears developer, you owe it to yourself to check it out today.

Monday, February 15, 2010

Python: Shuffling A Deck Of Cards

Python continues to amaze me on a regular basis.

I love programming in Python. It's simple to read, the code is clean, it's testable, it's just plain a pleasant experience. And, quite frequently, something happens that just makes me go "wow".

Wednesday, February 10, 2010

On Using The Wrong Tool

Subversion, as a version control system (VCS), is a mostly decent version control system. You check code in, you look at revisions, you look at logs, you can restore the state of your code at any time. It does the job it was built to do. This is not to say that it doesn't have problems, though. In fact, some of these problems have bitten me more than once. Using Subversion when other tools exist that actually correct these problems seems, to me, to be madness.

Tuesday, January 19, 2010

Announcing Availability of tgext.menu, v0.2

Long story cut short: I've not been moving on Bit Vizier as fast as I wanted for a number of reasons. One of them is the pain that is managing the navigation bar from a development perspective. If I want to move things around, I have to update my controllers and my navigation bar template. If I need to rename something, I have to do the same. If I want to add a feature, I have to do the same. The navigation bar, the side bar, and any other menus are all easily determined from the code. Why do I have to maintain a separate template?

With tgext.menu, I don't have to do so anymore. Instead of having a separate template, I simply have to provide a list of the menu entries, and the rest just happens. It makes dealing with things much easier.

tgext.menu is available on the Python Package Index (for use with easy_install), and also on bitbucket.org (to get the source code).

I'll apologize now for the shortness of this post. I'm still recovering from ear surgery, so mildly unavailable, but didn't want to hold this back any longer.

Wednesday, January 13, 2010

Choose The Right Data Structure

This one is actually almost embarrassing to post. I've been working on an extension for TurboGears called tgext.menu. I'll not get into long explanations of why, or what I hope to accomplish with it (I'll leave that for when the first release happens, hopefully early next week).

I will say this, though: If you don't choose the right data structure, right off the bat, you are condemning yourself to a hellish experience coding.

In my case, I had a list that looked like this:

  • ExitApp
  • Foo Spot
  • Foo Spot || Bar
  • Foo Spot || Baz
  • Foo Spot || Foo
  • Foo Spot || Sub || Bar
  • Foo Spot || Sub || Baz
  • Foo Spot || Sub || Foo
  • Sub || Sub 1
  • Sub || Sub 1 || Nested 1
  • TestHome
I wanted that to become a nested unordered list in HTML, looking like this:
  • ExitApp
  • Foo Spot

    • Bar
    • Baz
    • Foo
    • Sub

      • Bar
      • Baz
      • Foo

  • Sub

    • Sub 1

      • Nested 1

  • TestHome
Looks really simple, doesn't it? All you have to do is loop over the list, split each item on the || symbols, generate openings, put your entries, and generate closings. Except you have to compare the current base of the tree with the next base of the tree and the previous base of the tree.

I can't count the number of methods I tried, and every one of them had a small, subtle, error that threw off all the output. Sometimes elements would not be closed properly. Sometimes they would not be opened properly. I tried iterating, I tried recursion. All of them failed with errors that I could not find a generic way to correct.

Saying this was frustrating is a major understatement. Last night, I finally changed my data structure that represented the output: Instead of a list, I went with a tree. Using a tree, I was able to write up (in less than a dozen lines) a simple recursive loop that wrote out the entire tree, with appropriate whitespace, CSS class tags, the work.

In one hour, I solved problems that had been plaguing me for two weeks.

Lesson learned: If I'm spending a lot of time trying to solve a specific problem, and my algorithms are always producing errors, I need to look at the underlying data structures. I'll probably save myself a lot of time that way.

Wednesday, January 6, 2010

Are You So Arrogant ...

I was insulted last night. Now, there's nothing new about that, nor even anything particularly blog worthy. After all, we all get insulted in some fashion or another on a regular basis. We shrug it off, and we move on. This time, though, I've heard this particular insult too often. I want to address this, in particular: I was called arrogant.

Why? Because I answered a question.

The person who called me arrogant had asked about a particular piece of a program I've written, wondering what would happen if the user did a specific action. I replied that them doing so would be irrelevant, since it could do no harm. The reply came back, and amounted to "Are you sure?"

Suddenly, I wasn't sure. I went back to the code, and re-read it. I checked the actual behavior while running. I verified the contents of the database. Now, I was sure again: It simply didn't matter if the user did this action. It produced an off-by-one sum error for something which already has a much larger margin of error (counting cards that got sent and receieved from around the world). Being off by one in that situation causes no concern whatsoever.

Despite this, I then checked the database to see who was doing it, and found a whopping four people out of fifty three had done it. I spelled all of this out for the original questioner. Since the original questioner is not a technical person, I made sure to use terms that would make sense to her.

And then she asked the question that started with the title of this post: "Are you so arrogant ... " (yes, I've left the rest out, as I'm trying very hard not to make this an attack against her, and quoting her excessively would do just that).

I'm arrogant because I answered her question in terms that I knew she would understand.

After that, she mentioned that, in her experience, computer people (programmers, techs, administrators, etc) come across as if dealing with idiot children when dealing with non-computer people. So, instead of just being arrogant, I'm arrogant and condescending.

This is where the great disconnect occurs, and is the main point of this post: As a developer, I have to know things about the computer that many people do not. Doing web development, I have to know HTML, CSS, and JavaScript. If I'm doing anything with dynamic data, I also have to know whatever programming language I'm using on the server, and some database dialect so I can store, retrieve, and manipulate that data. All of that is the bare minimum required to do that work.

In order to do it well, I need to know some bits about the underlying operating system, the web server software being used, basics about HTTP (and possibly https), and the quirks of the various browsers in use around the world (IE6/7/8, Firefox, Opera, Chrome). Possibly more importantly, I have to have the knowledge that I can make no assumptions about anything. I can't count the number of times I've said "That's not possible to happen!" only to find out later that not only was it possible, it happened due to a bug in my code, and that bug came from an assumption.

In other words, I have to have a huge amount of specific, detailed, technical knowledge rolling around in my head just to do the basics of my job well. When I get somebody who is a self-proclaimed non-technical person asking me for answers that require at least some of that knowledge be handed out, I have to mentally change gears and use terms that I know will work to explain that knowledge. Of course I sound condescending, simply because I try very hard to make no assumptions.

The part that I find most interesting to me, in all of this, is one thing that my experience has taught me: The more understanding people have of their own systems, the less likely they are to call me arrogant when I explain an answer to their question.

I care a great deal about making sure I'm understood. There are times when my doing so comes across as rude, arrogant, or condescending. I do not set out to do so, but there are times when I cannot think of a way to avoid it. I actually don't like doing so. To have the insult lobbed in my direction on top of that is more than I can accept. I had to say something.

My final thought on all of this comes from her final remark: "Not everybody is a computer genius." That's correct, not everybody is. Not even me. If you want to find out about a few of them, allow me to give you some names to look up: Alan Turing, Steve Wozniak, Linus Torvalds, Alan Cox, Theo de Raadt, Brian Kernighan, Dennis Ritchie, Larry Wall, and Guido von Rossum. I am not, and never will be, in their league.