A Dropbox Replacement

So, I've just had an idea, and am not having much success finding it out there already. If you know of it, please let me know. What am I thinking of having it do?

This will need to do several things. First and foremost, it needs to support the same ideas as Dropbox: It syncs files across Windows, Linux, Mac, Android, and iPhone. It will allow people to share files with each other. It will also allow public sharing of files (such as photos).

The remaining pieces are where it gets interesting. I'd like the system to use a system like TrueCrypt. Using this, it will keep the entire dataset encrypted. It will also use many small files (on the order of 10M-100M), so that only an encrypted chunk has to be transferred, instead of the whole encrypted dataset. Doesn't sound like a big deal, until you start factoring in the ability to store photos. Imagine having to upload the entire dataset as one big chunk just because you rotated that picture of Uncle Jim where you were holding the camera sideways. 10G (or more) being uploaded, just for that, kinda sucks.

The backend storage is also different: Instead of uploading those chunks of data to a specific server somewhere, it uploads to a service utilizing Amazon's S3 protocol (obviously, Amazon itself is a viable choice for this).

As an added bonus feature, it should be able to commingle chunks from different S3 backends, and use that as a way to provide steganographic features. Using that, it would even be possible to have multiple layers of hidden volumes.

Another bonus feature: A destruct password. One that, when entered, will render the entire volume, and any hidden volumes, unusable. Considering the direction the government is going, this could be a very good thing to have.

An even better bonus feature would be having the TrueCrypt container dynamically resize as you add data, so that your space only goes as large as you need it to. With the idea of hidden volumes, this might not be feasible, but it would be nice.

Now, put all of this together, and you wind up with a service that is actually very easy to manage. It gives the security of TrueCrypt, the convenience of DropBox, and the mobility to use all of your data where you want and how you want.

Is this out there? Does anybody already have it available? Or is this something to be written at some point?

Wikis Suck

There, I said it. They suck. And it's not just wikis; pretty much every documentation system I've seen sucks. I'm not going to outline a solution here, because I don't have one. I can only say that these things are pretty awful, and until we can find something that actually solves these problems, we won't have a truly great documentation system out there.

I apologize for the nature of this post. It's a problem I've seen for years, and been trying to find an answer to. I just don't have one, and the problem is only getting worse. Maybe somebody out there can help to refine this problem into something that an answer can happen. In no particular order, here's the problems I've got with wikis:

Organization. I have yet to see something really handle this well. A straight up hierarchy doesn't work. Too often, a given document belongs in two places (or more). Anything that enforces a hierarchy must require a choice of where to place something. On the flip side, when discussing a wiki, very few of them impose any sort of hierarchy. In fact, some of them don't even have a hierarchy at all, placing every item in the same "folder". This makes for a mess when it comes time to search for something. Tags can help in this regard, but they're still far from ideal. How do we make sure that a given article is sufficiently tagged? How do we make sure that the searcher knows which tag will provide the desired results?

Authoritativeness. Sometimes, you need documentation that you know the source. You need to know it came from the right person or team. Documentation that is crowd-sourced simply doesn't always cut it. On the other hand, sometimes you need to have areas where the crowd can update things. This is a balancing act that is very difficult to get right, and I'm not convinced that the open model of the wiki is the right solution. Simply locking a contentious page doesn't actually solve the problem of determining accuracy, and the lack of accuracy in those cases undermines authoritativeness.

Access Control. When dealing with something open to the public, you're going to get spammers and defacers who don't care about what you have to offer. They're just going to come in and update every page they can with their garbage, and you have to clean up the mess. Some wiki systems simply go with "allow the user to revert to a previous version of the page", which is not actually useful at prevention.

Cross-Referencing. This seems like I'm talking about the primary strength of wiki: It cross-references so easily it's amazing. I can reference something anywhere (within the wiki, outside the wiki) without much effort at all. And yet, it's still a major weakness. Cross-references are entirely manual across all of these systems. If I'm reading a wiki, and decide to make a new page for a given phrase, I can find myself surprised that the page already exists. I have no way of knowing it unless someone else does the work of making the link.

Supported Formats. They support one specific type of media: Specialized markup text using the markup style of their choice. Why can I not upload a PDF, or DOC, or ODT? Hell, any other format. And I don't mean as an attachment: I mean as a first class piece of data to be reviewed, searched, and edited.

wikis are impressive, and do a difficult job. I'm not convinced they do it well, though, and I'm sure I'll have other issues to discuss about them later. I just wish I had a solution, instead of a rant.

The closest description I can have is that the ideal documentation system would be automatically cross-referencing, supporting a variety of data formats, with an intelligent search that would help me find what I'm looking for. It sounds like I'm asking for some sort of local internet with a Google search. Maybe I am, but I don't think so.

I just don't know. Does anybody else have any thoughts?

Announcing TurboGears 2.1.4 Release

The TurboGears team is proud to announce the release of TurboGears 2.1.4!

This release has many new features and bugfixes, all of them listed below. The most important one, though, is that this is the final 2.1.x release, and the final release that will support Python 2.4.

Please make sure to update your code to work on Python 2.5, as that will be the next supported Python version.

I would like to take a moment to thank Alessandro Molina especially. His work made this release possible, and as large as it is. He's been a great asset to the team, and I'm glad to have him on board.

• Deprecated Python 2.4 support. In 2.2, Python 2.4 support will be removed, and only 2.5-2.7 will be supported
• TurboGears extension architecture enhanced. Better support for hooks before and after configuration
• Performance enhancements in tg module and in default templates
• Enhancements to lazy_url
• WebOb version locked in. The change of virtualenv to "distribute" by default has broken dependency_links and allow_hosts in the config files, and this works around that issue.
• Jinja2 filters are now automatically loaded
• Work arounds for best_match, which was not producing the expected behavior with IE7 and IE8
• Add "auto_reload_template" as an .ini option
• Performance tuning the default size of the Genshi cache
• Added Genshi advanced i18n support
• Better compatibility with SQLAlchemy 0.7
• Changed default quickstart options to help ensure that some model is always available
• Documentation enhancements
• Nested RestControllers now work as expected (no longer required to implement "_custom_actions")

Worst Law Ever

What would it allow to happen? If you make a site that allows a user to post *anything*, no matter how small, and some user decides to post a link to a random youtube video which includes copyrighted music in the background, *your* site could be shut down.

This could even shut down Facebook, Google+, and Twitter. Possibly going so far as to shut down LinkedIn, too. Forums, Blogs, anything where any user is allowed to post content, can be completely shut down without viable recourse for the person being shut down.

How about Google groups, too? Those can be killed. Once even a link to copyrighted content appears, a copyright holder can demand the shut down of the site that has the link.

I am astounded. This might be the single worst piece of legislation I've ever even heard as regards copyright and the internet.

Here's a link to a better writeup, and includes some action points. Please, people, take them. Otherwise, the internet as we know it will be dead in this country, and the economic issues that we have right now are going to be called the good old days within a year.

Request For Ideas for Hiring Pond

So, a favor to ask of people: I'm working on a web application to help people manage their resumes. As I've gotten further in, I've realized I don't have an actual todo list for it. So, since I'm making this to be used by others, I'll ask everybody here:

What would you want to see? Currently, I've added enough code to allow the program to output something very close to my online resume ( http://www.icelus.org/ ). Next up, I have the following features on my todo list already:

• add docutils to output strings where appropriate (everywhere?)
• add login
• add ability to edit own resume
• add ability to compile new custom resumes
• add ability to output PDF
• add ability to output ODT
• add ability to output DOC
• add generic logo
• add generic photo / head (shadow) shot
• add ability to login using Google, Yahoo, Facebook, etc
• add ability to send resumes directly from Hiring Pond
• add ability to track where resumes have gone
• add ability to track job hunt progress
• add ability to display current employment status (employed, unemployed, contracting)
• add ability to display current employment seeking status (not at all, active, passive)
• add logo to qrcode
• change colors of qrcode
• change color scheme for all of hiring pond to sunset and water
• add fisherman silhouette as hiring pond logo

What else would you all want to see in order to make you want to use this?

"What Do You Do, Anyway?"

I get asked this quite a lot. What do I do? What value do I bring to a company? When asked, my answer is frequently "The impossible." I say this with a smile, and then explain "I do the stuff that everybody else says is too hard to do, can't be done, etc." and then go on to list some examples. If you want to see them, go check out my resume, and look at the projects at the bottom.

This answer, though, is very unsatisfying. It sounds like I'm bragging. It sounds like I'm saying I'm better than everybody else. It sounds, well ... it sounds arrogant. Since I'm on the job hunt now, I really shouldn't say this, but it needs to be said: I'm not that great, nor am I that special. I'm just a guy who happens to be passionate about his career, and doesn't shy away from some aspect simply because it doesn't fit with what I normally do.

This results in me doing such a variety of things that simply calling me by any one label is quite unrealistic.

Announcement: TurboGears2 2.1.3 Released!

We are pleased to announce the latest release of TurboGears, 2.1.3!

This release adds support for the Mongo object database and the Ming ORM for it, in addition to bringing in some small bugfixes. The full changelog is below

The number of changes is, admittedly, small, but don't underestimate the importance: NoSQL databases are an important tool for the web, and adding the support for Mongo brings TurboGears into the group of frameworks that not only supports it, but supports it well.

We're already looking forward to expanding support and closing even more issues for 2.1.4, so we'll see you next month for that release!

* Added support for MongoDB into the quickstart template
* Added lurl (lazy_url) support for late evaluation
* Adding tests module back into source distribution. Solves ticket 115
* Template caching is now manageable by TG applications
* Clean up output for flash() in default quickstart
* Performance enhancements
* Fixed missing dependencies in TG quickstart on Python 2.4