Thursday, February 7, 2013

A Distributed RESTful Architecture

Kaplan Flashcards - Developing an Architecture for Mobile Devices

Daniel Negri
Kaplan Test Prep


MCAT® Flashcards for iOS
Flashcards is a system, designed for mobile devices, that allows students to play with questions and answers. The application is based on scores of how many questions, represented as cards, the student is able to tag as "Known", "Unknown" or "Unsure" in a deck. Many projects at Kaplan stores data in different types of databases, including Relational Models, NoSQL or even plain XML files. The content is composed by exams like MCAT, LSAT, GRE, and students profiles. These applications place very different demands on storage, both in therms of content type, authoring, and presentation layout. In this article we start to describe the simple model provided by the RESTful Flashcards API architecture, which gives clients dynamic control over consuming data, presenting different layouts and formats. I describe the design and implementations of the architecture created to accomplish cross-platform mobile Flashcards clients.




Introduction


Over the last years we have designed, implemented, and deployed a distributed API system for managing structured data for Kaplan. The Flashcards API is designed to reliably scale to different sources of contents and thousands of machines. The Flashcards API has achieved several goals: wide applicability, scalability, high performance, and high availability. The API is used mainly by two Kaplan pilot projects, Flashcards Mobile and Flashcards Web, but is extended to custom authoring tools like the Flashcards Content Manager. The API clusters used by these products span a multiple range of configurations, that servers JSON to handle mobile native applications and browser based clients requests.

MCAT® Flashcards - Web Version

The Flashcards API provides a transparent interface to multiple content repositories performing content conversion, versioning, in-memory cache implementation. It provides a simple data model that supports dynamic control over data layout and format, and allows clients to reason about the locality properties of the data represented in the underlying storage. Data is indexed using users metadata, clients can control dynamically how to present it to the user.

Architecture Characteristics


The Flashcards architecture was created with coordinating a set of constraints that attempts to minimize latency and network communication while at the same time maximizing the independence and scalability of components. The Representational State Transfer (REST) fits very well once its became a consolidated style developed to accomplish all these points.

First Stage: A slice of an example database that stores questions and answers. The application receives multiple HTTP requests, it reads the content from a stand-alone database and return serialized JSON responses, asynchronously.


In a first stage we settled on this minimalistic model after examining a specific use-case, handle requests to different types of mobile platforms and browsers using a unified protocol.

We removed almost all business logics from the server, placing constraints on connector semantics. Important points moves from the server to the client like:

1. Page Rendering

That is, the logic to generate dynamically web pages or views, like JavaServer Pages (JSP), was moved from the server to the client. Today, with the advance of HTML5 and Javascript engines, it became very efficient render pages inside the client.

Most of times, clients utilize fluid layouts and customized components to present a better user experience. Aside the page rendering, a solid communication protocol  allows native Android, iOS applications to exchange data with the server.  

2. Sessions

Avoiding sessions leads to a more scalage and usable architecture. We are keeping the clients state on the client.

3. Caching

REST enables the caching and reuse of interactions. The server sets a group of HTTP Cache-Control directives. That is, it delegates the responsibility of caching to specialized systems like the browser or forward proxy.

Summary


This post makes part of a detailed group of implementations. I'll show how all components fit together and share more details on what's coming next. Stay tuned.

Monday, February 6, 2012

Startup - mkCRM - First Stage

A Open Source CRM for Mary Kay Consultants

At my first startup I created one open source social CRM software for Mary Kay consultants. Available online for free and inspired by SalesForce, Teambox, Spree. Supported by the open source community as well as the sale of our commercial products.

The Site




The Application





The Mobile Version

mkCRM Mobile is an HTML5 app that allows you get access to CRM APIs by multiple platforms like iOS, Android, Windows Mobile, Blackberry, and Symbian (we have plans to port for webOS and Bada too). I will put some pictures soon.


References

Link: http://www.mkcrm.com.br/
Source-code: https://github.com/danielnegri/mkcrm-rails 


Banking Monitoring System

Realtime Transaction Monitoring


Sicoob Monitoring Platform was released a couple years ago with a significant number of user interface changes. It’s always interesting to look at the direction that’s taken with the monitoring UI because it’s often used as a testing bed for future iterations of the Sicoob Platform user interface. I thought it would be fun to spend a weekend thinking about the Monitoring visual design and giving it a minor refresh.

The Monitoring Online System




The Consolidated Data System Analysis 




Interface complexity is an issue every engineer wrestles with when designing a reasonably sophisticated application. A complex interface can reduce user effectiveness, increase the learning curve of the application, and cause users to feel intimidated and overwhelmed.

I’ve spent the past year redesigning a particularly complex application with my primary focus being on reducing complexity.

Thursday, December 15, 2011

What Every Programmers Should Know About Floating-Point Arithmetic - Erlang

Erlang case - Why don't my numbers add up? 

So, open try this sample at your console:
$ erl
Eshell V5.7.4  (abort with ^G)
1> 0.1 + 0.2.
0.30000000000000004
You got a really unexpected result:
0.30000000000000004
Maybe you asked for help on some forum and got pointed to a *long article with lots of formulas* that didn't see to help with your problem.

Well, this site is here to:
  • Explain concisely why you get that unexpected result
  • Tell you know to deal with this problem
  • If you're interested, provide in-depth explanations of why floating point number have to work like that and what other problems can arise.
You must look at this site.

 Floating-Point Cheat Sheets for Erlang

Round Types

Value = 0.1 + 0.2.
0.30000000000000004
round(Value). % Will round to integer
0
round(Value * 10000000) / 10000000.
0.3
io:format("~f~n", [Value]).
0.300000
ok
io:format("~.14f~n", [Value])
0.30000000000000
ok

Tuesday, November 22, 2011

Polyglot Programming Motivation

Before
"Java came along in 1995, it was a welcome relief to C++ programmers tired of fighting with pointers, memory management, and other arcane plumbing. Java allowed them to just get work done".
The designers of Java purposely made the Java language look and feel a lot like C++ just because for Java to succeed, it need to appeal to the current high priesthood of developers, those same C++ developers.

Today
"In ten years, everyone will be programming in Smalltalk, no matter what they call it".
- Glenn Vanderburg

Backward compatibility no longer makes much sense. Java developers must learn some strange stuff that has nothing to do with getting real work done but rather with strange rituals. Think about this code:

    public class HelloWorld {
        public static void main(String[] args) {
            System.out.println("Hello, Wold");
        }
    }


Remember such a zero-based arrays, as distinction between primitives and objects. Java is rife with C++-inms and baggage that don't help the productivity of modern developers, all that made sense in 1995.

Good Chance for Changes: separated language from platform
"Fortunately, the designers of Java made a brilliant decision when they built Java: they separated the language from the platform. That gives developers a "get out of jail free" card called polyglot programming".
Developers can use specialized languages to solve specific problems, we now have hundreds of languages that run on the Java virtual machine and the .NET managed runtime.

The idea of polyglot programming is different of use SQL to access databases, JavaScript to add interactivity to web pages. These examples are orthogonal to the JVM: they run outside the world of Java. And that causes big headaches. It is about leveraging languages that produce bytecode within the JVM, eliminating impedance mismatch.

The other impediment to polyglot programming is the perception that you are changing languages. Always in the past, changing languages meant changing platforms, which has an obvious bad connotation to developers who don't want rewrite all their libraries again. But the separation of the platform from language in both Java and C# means you no longer have to fight that battle. Polyglot Programming allow you to leverage all your existing assets but also use languages more suitable to the job at hand.

Examples

Reading Files the Groovy Way (full example)

def number=0
new File (args[0]).eachLine { line ->
    number++
    println "$number: $line"
}


JRuby and isBlank (full example)

class String
    def blank?
        empty? || strip.empty?
    end
end

  
class BlankTest < Test::Unit::TestCase
    def test_blank
        assert "".blank?
        assert " ".blank?
        assert nil.to_s.blank?
        assert ! "x".blank?
    end
end

Jaskell and Functional Programming (full example)

new SafeArray arr = {
    length = len;
    at i = if i < begin || i >= len then
              throw $ ArrayIndexOutofBoundsExeception.new[i]
           else
              arr[begin + i];
}


Future

The days of trying to cram solutions into a single language are disappearing. Because we have outstanding managed runtimes - Java and CLR - we should leverage with better tools. Polyglot programming allow you to mix and match solutions without discarding all the code you already have that does important work. Language development is exploding on these two proven platforms.

As developers, you need to learn how to take advantage of this growth so you can write better code using tools more suited to the job.


Further Reading

Neal Ford, Meme Wrangler
The ThoughtWorks Anthology: Polyglot Programming,
Pragmatic Bookshelf, 2008
Buy from Amazon