How to Minimize IT Exam Lameness

In my opinion, it is possible to test expertise in computer languages and tooling in better ways than for instance JavaBlackBelt does.
Exam questions in JavaBlackBelt and comparable systems tend to be
  • multiple choice
  • answerable by searching the documentation or online forums
  • answerable by copy-paste-compilation
  • "mined" by tiny syntax errors

The way I would test knowledge:
  1. present a UML schema of a system
  2. let the answer be a region of grid cells within the image or a set of related entities/connections in the architecture
  3. ask where the system is least efficient, contains an anti-pattern, is overly complex or restrictive, or some other question requirying non-online-searchable insights
Because what I like to know is:
  • do you know your ultimate goal when you automate something?
  • are you familiar with idioms, patterns and anti-patterns? (not reinventing the wheel)
  • can you forecast program usage, user issues and refactoring steps?
  • do you have an eye for bottlenecks in system configurations? - do you think small or think big, using minimal resources (mobile app) versus building without limits (cloud service)
  • how do you go about building solutions (human + software) that CAN guarantee outcomes, no matter how imperfect the underlying resource
  • are you a human symbolic logic parser :) - can you crop the meaningless (in design/requirements docs)
  • do you have the mindset for fearless and natural paradigm shifts
  • have you got a feel for when to break the rules

    Using the Strengths of People and those of IT

    In my opinion, there is only one main question when solving problems through computers. Before I tell you what I think that question is, let us first define IT as two fases:
    1. distilling the essential from the information stream
    2. building a generic solution that will remain useful

    Fase 1 depends on recognising the abstract behind the deluge of data. A person experienced in defining requirements, seeing patterns and creating control points (parameters) can do this better than any automated system.
    The automated result of fase 2 should save a lot of time, attention and nerves compared to manual inspection and actions. A machine parsing symbols, applying well-defined rules and crunching numbers can do this much faster than any human.

    So, proper application of IT is using the power of people combined with the power of computers. The human should analyse the problem and define the template and parameters. The machine then carries out calculations at super-human speed. When the program cannot handle a configuration, we go back and exclude (or maybe include) that exception.
    Sometimes the problem at hand requires common sense and practical innovation that solves only the core issue.

    When there is time, it pays off to look at the problem sideways, and deliver a set of templates for future solutions. Here are a couple of examples from my experience with Fase 2:


    Generating XHTML + CSS + SVG + patterns/backgrounds, then finetuning

    Fast set up for the web, freeing time for finetuning design. And you get to play with Scalable Vector Graphics, minimizing transfers of binary images (but watch out for browser-specific SVG quirks).
    Next step: a styling validator for checking golden rule proportions, colour matching and readability of a web page.


    Document Generation through XSLT

    I built a structured set of XSL Transformation that received XML input (from a financial business framework) and created PDF output. The document templates + database containing variable data + transformation scripts can produce output almost instantly. The only weak points can be the XML structure (checks on the structure can be automated) and incorrect data (checks on the data can be automated to a degree).


    Custom Automation within CRM

    Working with the combination of Microsoft CRM, SQL Server (+ Reporting Services) and Sharepoint, one often has to choose to implement a manual fix to a problem, or to set up a generic solution. For instance, in CRM it is possible for users to create workflows that automate checks, actions, and warnings. Customizations of entities can be exported to XML, either in order to extract the definition of the system, or to change definitions of fields and filters in semi-automated way. As soon as we have the system definitions in XML, we can parse and change it automatically.


    Controlling 3D Generation through Parameters

    I wrote a VRML generator in C++ that non-IT users could direct via parameters. The goal was to create urban environments from scratch that were geometricaly valid and acceptable to the human eye (as realistic). Ultimately, my aim was to replace the waterfall-like normal 3D modelling process for non-specific content.
    The modellers would create or acquire specific content such as a statue, and then complement this by choosing out of a wide array of generated 3D. Directing this generation would be a matter of trying out different sets of parameters and choosing results based on visual inspection, a much more predictable process than depending on creativity while under time and budget constraints.

    The reality of IT is often working under the contraints of frameworks, time, budget, and agreements. These constraints often exclude any real generic solutions, and so efficiency is reduced to human manual action speed. Which can mean that the net effect of an IT system is only 10% higher efficiency, when it could be 200%.
    In order to achieve more generic solutions and align business and IT it is beneficial to keep asking: how can we harness the main strength of humans and the main strength of computers?