Showing posts with label couchdb. Show all posts
Showing posts with label couchdb. Show all posts

Sunday, July 14, 2013

Discovering JavaScript Closures

One of the biggest issues I have with JavaScript are callback functions which cannot access data from their parents scope. I tried OOP to get around this problem, yet the lack of a reliable $this variable voids those efforts pretty much.

Right now I'm working on a piece of middleware which should download a file from a CouchDB server, send it to a service providers API and obviously update the source document with the result.

Sounds easy, but if the callbacks don't know about the CouchDB document any longer this simple task quickly becomes a nightmare.

And so it happened that I've discovered this article about JS closures. Not entirely sure how it works under the hood but it sure solved my problem. The final code looks a bit like this:

function change_processor(change, param) {

  return {

    download: function() {

      for (var filename in change.doc._attachments) {                                               
        http.get({
          url: param.dbhost,
          port: param.dbport,
          path: '/' + param.dbname + '/' + change.doc._id + '/' + filename
        }, function(res) {
          var data = [];
          var dataLen = 0;
          res.on('data', function(chunk) {
            data.push(chunk);
            dataLen += chunk.length;
          }).on('end', function() {
            var buf = new Buffer(dataLen);
            for (var i=0, len = data.length, pos = 0; i < len; i++) {
              data[i].copy(buf, pos);
              pos += data[i].length;
            }
            console.log(buf.toString('ascii'));
          });
        });
      }
    }
  }
}

module.exports = change_processor;

The cool thing about this closure is that variables like `change` are available throughout the entire construct, including the callbacks of http.get. It's incredible.

Wednesday, July 10, 2013

Why using CouchApps?

Seeing one of CouchDB's coolest features, CouchApps, being neglected by most CouchDB users often saddens me. While CouchDB itself makes for a great noSQL data storage it may also be used to serve web applications with excellence.

CouchApps are nothing but design documents in a CouchDB. They may only contain stuff like map/reduce functions or replication instructions. Yet they may also hold any number of assets like css files, images, videos and so on. On top of that, the document itself is a json document that can be accessed in a common way.

CouchDB's internal http server can then serve those files and data with ease. Data may be displayed and managed by nothing else than the CouchApp itself, eliminating the need for an additional layer all together.

There seems to be a mind problem with that out there in the community. To me it appears almost as if people can't let go of the idea to separate data storage and application. Yes, CouchApps have some limitations to what you can do with them but in most cases those limitations don't get in the way of what the apps are doing.

So by adding another application layer (e.g. a PHP script) all developers really do is to add more work. CouchDB was designed to make life easier. It's ability to serve entire apps is one of the biggest steps in that direction so you should consider making use of it.

To get started, make yourself familiar with Design Documents and all the bits and pieces they may contain. Of course there is a number of tools making life easier.

Even so it's no longer maintained the python tool couchapp still does a fantastic job. Besides the website contains a lot of nice insights into how CouchApps work.

Then there's kanso. Like the couchapp tool it allows you to structure your CouchApp within the file system of your computer. But it does a lot more than just that. Kanso introduces a node.js stype package manager to the world of CouchApps.

Every kanso app is a package and can be reused by another App. That makes it extremely nice to work with in terms of modular design. Imagine a number of apps all of which belong to the same overall project. Another app contains global code and assets like templates, configs, etc. The different apps may then use that "core app" and you can reuse every bit of code in there.

However, each app is completely atomic. You may use different core versions within them so that they can live on their own. Whenever you change something for one app that doesn't necessarily effect all the others. That's good news if you want to avoid adding more problems by solving bugs within one feature.

To me kanso is one of the best choices for webapp development. It's fast and the modular approach makes sharing code a no brainer. I seriously suggest you give it a go and find out how you might save a ton of development time by cutting out the middle man to relax a little more.

Tuesday, July 9, 2013

Getting things done with JavaScript

I've been a software developer for more than 15 years now. There have been lots of ups and downs. It feels fantastic to label a project "done" after month of hard work. At other times I wished for another job and really hated everything about development.

It's the lengthiness of projects that made it increasingly difficult to love with my work. But all of that changed just recently.

I wrote a lot of software using PHP or Perl. Those who know me know that I don't limit myself to those languages at all but that's what sold best. Yet despite my wealth of experience with a number of programming languages I never managed to develop my own product.

Still that's what I want most. I know that I have only so much energy left after my daily work is done. Asking a developer to work for eight hours a day is just not realistic. Squeeze four hours out of a creative programmer and you're lucky.

Building my own product on the side however takes a lot more than that. After my eight hours jobs are over I'd still have to put in a few more hours every day. For a while that's doable but not in the long run.

So how to solve this problem? Simple, use modern technology rather than the common patterns you're used to. PHP does a fine job if done right. Use a framework like CakePHP or CodeIgniter and you'll save tons of time in development. Yet still it takes quite a bit of time to get basic issues like user logins right for each individual project. SQL isn't really helpful either and even so there are module systems like PEAR I don't really like using them.

I consider those languages, as good and useful as they are, old world languages. The day I discovered JavaScript for me was a revelation and these days it's no longer limited to the browser. Node.js is a great example. A lot of common tasks are handled by modules that are easy to manage and are pretty common to use.

Attach that to a noSQL database like CouchDB or MongoDB and you've cut down development time significantly. Just recently I discovered the kanso famework allowing me to reduce my stack even further. I've played with CouchDB and CouchApps a while ago yet it's kanso, that really got me excited.

CouchApps live wihtin CouchDB itelf. They're written in JavaScript and require only little code to get Webapps running. There's no need for SQL, kanso provides a great module system similar to node.js and almost the entire app is written in frontend code.

I'm impressed on how fast I'm able to develop apps now. A lot of the basics just work and I can focus on what matters most. Armed with these new world tools I'll sure be able to crank out a product in what little time I have left.

Sunday, July 7, 2013

Creating a private kanso package repository

Setting up a private kanso repository is an easy task and it makes a lot of sense. I consider using kanso and CouchDB for a rather large project which should be split up into many smaller packages. That way I can make use of kansos package mechanism and don't have to repeat myself when it comes to code shared by all the different apps.

Of course I could easily use the public package repository and whenever an app is generic enough it should actually be used. But when your package is highly specialised or not meant to be published having a private repository gets you around that problem.

The first thing you need to do is setting up the repository itself. Check out the Github Project and install it to a CouchDB server of your choice using kanso. And that's really all there's to it. It's amazing how simple it is.

Now use the --repository or --repo (that's not consistent yet) parameter to specify your local repository. Here's an example:

kanso publish --repo http://localhost:5984/kansorep/

In your final step you need to edit `lib/kansorc` within your kanso installation. Find the line that says `export.DEFAULTS` and add your repository URL. Here's an example:

exports.DEFAULTS = {
    repositories: [
      "http://kan.so/repository",
      "http://localhost:5984/kansorep"
    ],
    env: {
        // custom push locations
    }
};

Once that's done, kanso is smart enought to look for packages within all both repositories. That means that you'll be able to use `kanso install` without having to worry where the packages you're looking for are located.

I've gotta admit, that the repository handling of kanso isn't great yet. There's an old discussion touching that subject. Yet it doesn't look like that the functions suggested in there have been implemented yet.

Saturday, July 6, 2013

Kanso Build Processors

Today I've created a custom jQuery package for Kanso. It's not only containing a much later jQuery version but also all the jQuery extensions I normally use. My goal was to compile all of them into one single file so that applications using it don't have to contain a column of script tags just to get all the extensions in.

Initial effort

I begun by creating a new package with a `kanso.json` file like this:

{
  "name": "jquery2",
  "version": "2.0.3",
  "categories": ["utils"],
  "attachments": [
    "jquery.js"
  ],
  "maintainers": [
    {
      "name": "Arthur McFlint",
      "url": "https://github.com/arthurmcflint"
    }
  ],
  "url": "http://jquery.com/",
  "dependencies": {
    "attachments": null,
    "modules": ">=0.0.8"
  },
  "description": "Contains the jQuery core as well as some extensions used commonly used by my projects."
}

By adding the `jquery2` extension to an app it automatically adds jquery.js.

Fine Tuning

Even so the above solution works like a charm it has a massive flaw. I couldn't care less about manually maintaining one big file containing the jQuery core as well as the extensions. After a while it would be beyond repair.

Instead I added build processors and some JavaScript files executed by node when running kanso. The new `kanso.json` file looks a bit like this:

{
  "name": "nexus-jquery",
  "version": "2.0.3",
  "categories": ["utils"],
  "attachments": [
    "jquery.js"
  ],
  "maintainers": [
    {
      "name": "Arthur McFlint",
      "url": "https://github.com/arthurmcflint"
    }
  ],
  "preprocessors": {
    "merge": "build/merge"
  },
  "postprocessors": {
    "cleanup": "build/cleanup"
  },
  "url": "http://jquery.com/",
  "dependencies": {
    "attachments": null,
    "modules": ">=0.0.8"
  },
  "description": "Contains the jQuery core as well as some extensions used commonly used by my projects."
}

Whenever kanso is pushing an app to the server it first goes through the process of building the app. By adding 'hooks' in the shape of preprocessors and postprocessors it's possible to alter the standard behavior quite a bit.

First of all I've created a new file calles `build_config.json`:

{
  "jsfiles": [
    "jquery-2.0.3.js",
    "extensions/misc/jquery.ba-serializeobject.js"
  ]
}

Then I've added `build/merge.js`:

var uglify = require('uglify-js');


module.exports = function(root, path, settings, doc, callback) {

  // Read the extensions config file                                                                                                                                                                   
  var build_config = JSON.parse(fs.readFileSync(__dirname + '/../build_config.json', 'ascii'));

  // Fix pathes to js files                                                                                                                                                                            
  var jsfiles = [];
  build_config.jsfiles.forEach(function(path) {
    jsfiles.push(__dirname + '/../' + path);
  });

  // This is where the output will go to                                                                                                                                                               
  var target_file = __dirname + '/../jquery.js';

  // Init output buffer                                                                                                                                                                                
  var output = '';


  // Combine JS files into one string                                                                                                                                                                  
  jsfiles.forEach(function(path) {
    output += fs.readFileSync(path, 'ascii');
  });

  // Add newline at the end of file                                                                                                                                                                    
  output += "\n";

  // If compression is active, minify the code                                                                                                                                                         
  if (settings.jscompression) {
      output += uglify.minify(output, {fromString: true})
  }

  // Write jquery.js                                                                                                                                                                                   
  fs.writeFileSync(target_file, output);

  // Carry on :)                                                                                                                                                                                       
  callback(null, doc);
};

And of course `build/cleanup.js`:

var fs = require('fs');

module.exports = function(root, path, settings, doc, callback) {

  fs.unlinkSync(__dirname + '/../jquery.js');
  callback(null, doc);

}

To make it all work, I've finally added a `package.json` file adding `uglify-js` to the requirements and installed it with npm.

The result

As a result the code is now automatically merged from the files I've added in the `build_config.json` file. If I turn on `jscompression` in the `kanso.json` file of my main application using the jquery2 package, kanso automatically minifies the code during the build.

All in all that's a very nice result and shows how much tweaking is possible within the kanso framework. The only thing you have to be cautious about is async code. I ran into big trouble using minify or node-minify to do the job. However as long as the functions run in sync mode you're good to go.