TimesTen


TimesTen is an in-memory, relational database management system with persistence and recoverability. Originally designed and implemented at Hewlett-Packard labs in Palo Alto, California, TimesTen spun out into a separate startup in 1996 and was acquired by Oracle Corporation in 2005.
All data within a TimesTen database is located in physical memory, which means that no data operation requires disk I/O. This is unlike traditional disk-optimized relational databases such as the Oracle Database, DB2, Informix or SQL Server, whose designs must contain algorithms that attempt to minimize disk accesses. TimesTen provides applications with short, consistent response-times and very high throughput as required by applications with database-intensive workloads.
As memory operates far faster than hard disk, TimesTen is used in applications where service-level agreements require low and predictable response times, such as telecommunications, real-time financial-services trading applications, network equipment, and large web applications. Also, unlike other memory-caching systems that use key-value pairs, TimesTen can be accessed with standard interfaces and provides the functionality of the SQL query language.
Applications with data residing in an Oracle Database can utilize TimesTen through a database caching option,
in which TimesTen functions as an in-memory cache database in front of an Oracle Database.
TimesTen runs on most major Unix/Linux platforms and on various Windows platforms, in both 32-bit and 64-bit modes.

Technology

TimesTen is an in-memory database management technology that provides very fast data access time. It assumes all data will reside in physical memory during run time. The result is very low response times, which enable high throughput, even on commodity hardware.

Architecture

Shared libraries

TimesTen functionality is contained in a set of shared libraries that application developers link to their application, allowing TimesTen to execute as part of the application's process. This shared library approach is different than conventional RDBMS systems where the database is essentially a set of distinct processes to which applications connect via some form of inter-process communication. This communication may take the form of a client/server connection spanning over a network or it may be some form of intra-system IPC such as a Unix domain socket connection or a shared memory based connection mechanism.

Memory-resident database

The data for each active TimesTen database is stored in a shared memory segment, allowing multiple TimesTen databases to be active concurrently, and also allowing an application to simultaneously access several TimesTen databases on the same system. On 64-bit platforms, the size of a TimesTen database is practically limited only by the amount of RAM available on its host computer. One customer has a production TimesTen database approaching two terabytes in size utilized for on-line transaction processing.

Database processes

Starting TimesTen requires starting a background process called the TimesTen main daemon, which then starts multiple TimesTen subdaemon processes to manage each database created in the system. These subdaemon processes perform database operations such as:
  1. loading and unloading the database into RAM
  2. writing periodic fuzzy [|checkpoints] of the TimesTen database to disk
  3. writing transaction log records from the in-memory log buffer to the [|transaction log files] on disk
  4. handling deadlocks

    Connection modes

Client applications that connect to traditional disk-based relational databases typically use TCP/IP or another IPC mechanism to communicate with a database server process. In TimesTen, applications that reside in the same server as the TimesTen database can connect directly to the in-memory image of the database by using the TimesTen direct driver, eliminating the need for any inter-process communication of any kind, thus providing extremely fast performance. If the application resides on a remote server, the application can also connect to the TimesTen database using the traditional client/server model of data access.

Availability and integrity

Checkpoint and transaction log files

All TimesTen data exists in RAM, however TimesTen does utilize non-volatile storage for database persistence and recoverability. A TimesTen database stores all transactional data modifications in an in-memory log buffer, which is eventually persisted to disk in the form of transaction log files. In addition, TimesTen also persists snapshots of the in-memory database, called checkpoint files, to disk. The combination of checkpoint files and transaction log files allow TimesTen to provide recoverability in the event of a system failure. TimesTen implements a parallel log manager in order to maximize throughput on large SMP systems.
By default, TimesTen operates in non-durable commit mode. In this mode, a commit operation occurs purely in memory, and the writing of the log records for the transaction to disk occurs asynchronously to the commit. This provides for very low response times and very high throughput at the cost of the potential for some small amount of data loss in the event of a system failure. A true synchronous commit mode is also provided; this mode avoids the possibility of any data loss at the cost of reduced performance. When operating in synchronous commit mode, TimesTen provides automatic group commit optimization. TimesTen allows the architect / developer to balance performance versus data safety by providing control of the commit mode at three different levels: database, connection, and transaction. Another option for data protection and high-availability is to use TimesTen replication.

Replication

The TimesTen replication mechanism enables a highly available system by sending database updates between two or more hosts. With TimesTen replication, a master host sends updates to one or more subscriber hosts. TimesTen recommends an active-standby pair configuration for highest availability. An active-standby pair consists of two master databases, an active and a standby database. In addition to the active and standby databases, multiple subscriber databases can be configured to serve as disaster recovery copies or read-only farms.
Replication in TimesTen is implemented by the replication agent, a daemon process that aims at every TimesTen database involved in replication. The replication agent sends and receives updates between databases by communicating with the replication agents of other databases involved in the same replication scheme. Change capture is via log-mining with in-memory optimization; under normal operating conditions change records are captured from the in-memory log buffer with no need for any disk I/O. Communication between replication agents is via an efficient stream-oriented protocol built on top of regular TCP/IP connections. The replication agent is multi-threaded and in the 11.2.2 release, TimesTen supports parallel replication for increased throughput. The replication workload is automatically parallelized to maximize throughput while still maintaining correctness.

In-memory database cache

TimesTen provides the capability to cache data from an Oracle Database source. When used in this fashion, TimesTen is referred to as Oracle In-Memory Database Cache , an Oracle Database product option.
To utilize Oracle Database caching, one defines one or more SQL objects known as cache groups. A cache group is a set of one or more related database tables and allows for subsets of its rows and/or columns. Database tables in a cache group must each have a defined primary key or a unique index declared across a set of non-nullable columns and must be related in a parent-child hierarchy via primary key-foreign key constraints. SQL predicates can be used to control what data is to be cached.
Once a cache group is defined, the cache group can then be "loaded", allowing Oracle Database data to be cached in TimesTen. Applications can then read from and write to cache groups, and all data modifications will then be synchronized with the corresponding Oracle database tables either automatically or manually.

Event notification

TimesTen provides an event notification mechanism through the transaction log API. XLA provides functions which allow applications to be notified of data changes occurring in TimesTen tables. XLA can also be used in conjunction with materialized views to simplify the process of monitoring changes made to rows spanning multiple tables. XLA works by mining the TimesTen transaction log stream in a similar manner to the replication agent.
Database application deployments that do not include an Oracle database which want to use TimesTen as an in-memory cache database can use XLA to capture updates and, via custom application code, apply these updates to their backend database.

History

TimesTen was founded in HP labs by Marie-Anne Neimat, Sherry Listgarten, and Kurt Shoens, under the name of "Smallbase". At HP, Jean-René Bouvier decided to embed Smallbase into HP OpenCall, which made the first commercial use of the product in 1995.
In 1996, the product was spun off into a separate venture capital funded startup company based in Mountain View, California under the leadership of CEO Jim Groff. The product became popular for telecommunications equipment, as response times in the milliseconds or even microseconds were required for applications like packet switching. The company had 90 employees and was profitable when it was acquired by Oracle Corporation in 2005. After the acquisition, Neimat remained as the director of TimesTen development at Oracle, adding many Oracle database features to the product such as support for PL/SQL and integration with Oracle SQL Developer and Oracle Enterprise Manager.