Developer world

How a Serializable isolation level behaves

2007-11-29T16:22:00.000-08:00

When using Oracle, or other MVCC (Multi Version Concurrency Control) databases like Postgresql or MySql in combination with InnoDb, a transaction with the SERIALIZABLE isolation level doesn’t always block (a pessimistic approach) when it conflicts with another transaction. Instead a more optimistic approach is used. This approach however can lead to an abort of the transaction with the feared: “ORA-08177: can’t serialize access for this transaction”, which in essence is just an optimistic locking failure.

A SERIALIZABLE transaction in classic lock based databases, is implemented by using pessimistic locking. The transaction needs to obtain the exclusive table lock prior to accessing the table. This prevents other SERIALIZABLE transactions and non SERIALIZABLE transactions (these need to obtain the shared table lock) from executing concurrently. The consequence is that a SERIALIZABLE transaction blocks until the exclusive table lock is obtained, or when it is rolled back (eg a timeout, detection of a deadlock, etc).

In Oracle no exclusive table lock is required to implement the SERIALIZABLE isolation level. Instead Oracle is able to take a snapshot of the database and use this snapshot for the duration of the transaction (transaction level read consistency). This is realized by reading the SCN (the System Change Number: a version number that is increased when a transaction commits) when the transaction begins and using this SCN and the data in the rollback segment, to reconstruct previous versions of records. The consequence of this approach is that different transactions could see different snapshots at the same moment (hence Multi Version Concurrency Control). Therefor a SERIALIZABLE transaction is not able to see other updates (so no worries about unrepeatable reads) but also not other inserts/deletes (so no phantom reads).

In short: instead of relying on pessimistic locks to prevent isolation problems, the transaction is able to see the state of the database as it was when the transaction began, even though other transactions could have changed the database in the meanwhile! This is one of the reasons why Oracle is able to perform well under heavy load.

There is one caveat: most transactions do more than just reading (else the READ_ONLY isolation level would have been a better solution). When a transaction modifies (inserts, updates, deletes) records, these records are exclusively locked for the remaining duration of the transaction (these changes are visible inside the transaction of course), independent of the isolation level. But once a transaction ends, these locks are released and without extra protection, it could happen that a SERIALIZABLE transaction updates an older version of the record (a reconstructed one) than the last committed version. This problem is called a lost update.

Luckily Oracle protects us from lost updates with SERIALIZABLE transactions by adding an optimistic lock: when the transaction tries to update a record that has been committed after the transaction began, it knows that a different transaction has modified it and the transaction aborts with the “ORA-08177: can’t serialize access for this transaction”. Just like optimistic locking failures in normal database communication, the standard policy of dealing with this situation is to retry the transaction and hope that the transaction isn’t interfered again.

Performance IS NOT Scalability

2007-11-18T03:40:00.000-08:00

It’s funny how one of the key things I needed to remove to make my site usable is one of the first things someone else recommends adding to make Drupal sites usable - and we’re both right.

RealJenius.com for all intents and purposes is a low-traffic site. It’s certainly more than most of the sites I host for small businesses or for my family, but all-in-all, the traffic and load is low.

I suffered at my last host for many reasons, as is well documented here, but one such reason was the communication time between the database and my web server was just too high. My current configuration of RealJenius.com favors several queries, and even with caching enabled, it needs a fairly efficient transfer from the DB to the web server.

When I moved to a MediaTemple DV, my SQL server and Web server became one in the same, and now communication between MySQL and PHP is lightning-fast. Granted, this is compounded by the fact that my server is much less overwhelmed than the piece-of-garbage I was on at nightmarehost.

John & Cailin have a very good article on the steps to scale a Drupal install to higher and higher demand - and their first recommendation is to move the database off of the web server. Of course, this configuration makes sense, because if they are sharing the same hardware, they are sharing several resources, including memory, CPU cycles, and IO cycles.

The most scalable configurations, however, will also carry with them some fixed costs. There is a natural overhead in technologies like clustering, load balancing, and networked decoupling; in many ways similar to the constraints applied by Amdahl’s Law. You can’t just double the servers, and expect double the performance.

Of course, the same differences can be applied to programming - the fastest algorithms for reasonably small sets may be very resource-hungry, using gobs and gobs of memory; the fastest data structure sorting and searching can be done when the whole data tree is in memory for example. However, if that structure gets large enough, you can quickly run out of resources, exposing the fact that the algorithm is not very scalable.

It’s up to the developer to decide the trade-offs at this point - how much data is a reasonable amount, and how much scalability do you really need.

You often pay a cost for scalability, and I was paying the cost for my host’s need for scalability, and they weren’t doing anything to relieve the performance headaches their ‘scalable’ architecture was creating for their users.

Google OpenSocial: after the hype, the holes

2007-11-12T14:20:00.000-08:00

Open standards always cause security problems and Google's OpenSocial API introduced last week is no exception. Not only was an early application based on the standard hacked within minutes, it quickly became evident that OpenSocial is vulnerable and offers an open door to anyone who wants to put a little effort into pushing it open.

But it is not only OpenSocial's lack of security protection that makes it vulnerable. First, OpenSocial is not an "open" standard at all. It is a proprietary API defined by Google's commercial priorities - including a transparent knee-jerk response to Microsoft taking a stake in the acknowledged market leader Facebook. The history of IT is littered with unsuccessful attempts to promote proprietary standards as "open" (just look at IBM with SNA and SAA).

Second, there is already a genuine open standard (Openquabal) - supposedly designed to do the same things as the OpenSocial API. Unfortunately Openquabal does not have the backing of a predatory multinational corporation so it will probably fall by the wayside.

Then again, large corporations spent most of the 1980s and 1990s bickering about common networking standards only to be outflanked by a 30-year old academic standard devised by the US Department of Defence. It was called ARPANET.®

15 Coolest Firefox Tricks Ever

2007-11-08T13:18:00.001-08:00

Everybody’s favorite open-source browser, Firefox, is great right out of the box. And by adding some of the awesome extensions available out there, the browser just gets better and better.

But look under the hood, and there are a bunch of hidden (and some not-so-secret) tips and tricks available that will crank Firefox up and pimp your browser. Make it faster, cooler, more efficient. Get to be a Jedi master with the following cool Firefox tricks.

1) More screen space. Make your icons small. Go to View - Toolbars - Customize and check the “Use small icons” box.

2) Smart keywords. If there’s a search you use a lot (let’s say IMDB.com’s people search), this is an awesome tool that not many people use. Right-click on the search box, select “Add a Keyword for this search”, give the keyword a name and an easy-to-type and easy-to-remember shortcut name (let’s say “actor”) and save it. Now, when you want to do an actor search, go to Firefox’s address bar, type “actor” and the name of the actor and press return. Instant search! You can do this with any search box.

3) Keyboard shortcuts. This is where you become a real Jedi. It just takes a little while to learn these, but once you do, your browsing will be super fast. Here are some of the most common (and my personal favs):

Spacebar (page down)

Shift-Spacebar (page up)

Ctrl+F (find)

Alt-N (find next)

Ctrl+D (bookmark page)

Ctrl+T (new tab)

Ctrl+K (go to search box)

Ctrl+L (go to address bar)

Ctrl+= (increase text size)

Ctrl+- (decrease text size)

Ctrl-W (close tab)

F5 (reload)

Alt-Home (go to home page)

4) Auto-complete. This is another keyboard shortcut, but it’s not commonly known and very useful. Go to the address bar (Control-L) and type the name of the site without the “www” or the “.com”. Let’s say “google”. Then press Control-Enter, and it will automatically fill in the “www” and the “.com” and take you there - like magic! For .net addresses, press Shift-Enter, and for .org addresses, press Control-Shift-Enter.

5) Tab navigation. Instead of using the mouse to select different tabs that you have open, use the keyboard. Here are the shortcuts:

Ctrl+Tab (rotate forward among tabs)

Ctrl+Shft+Tab (rotate to the previous tab)

Ctrl+1-9 (choose a number to jump to a specific tab)

6) Mouse shortcuts. Sometimes you’re already using your mouse and it’s easier to use a mouse shortcut than to go back to the keyboard. Master these cool ones:

Middle click on link (opens in new tab)

Shift-scroll down (previous page)

Shift-scroll up (next page)

Ctrl-scroll up (decrease text size)

Ctrl-scroll down (increase text size)

Middle click on a tab (closes tab)

7) Delete items from address bar history. Firefox’s ability to automatically show previous URLs you’ve visited, as you type, in the address bar’s drop-down history menu is very cool. But sometimes you just don’t want those URLs to show up (I won’t ask why). Go to the address bar (Ctrl-L), start typing an address, and the drop-down menu will appear with the URLs of pages you’ve visited with those letters in them. Use the down-arrow to go down to an address you want to delete, and press the Delete key to make it disappear.

User chrome. If you really want to trick out your Firefox, you’ll want to create a UserChrome.css file and customize your browser. It’s a bit complicated to get into here, but check out this tutorial.

9) Create a user.js file. Another way to customize Firefox, creating a user.js file can really speed up your browsing. You’ll need to create a text file named user.js in your profile folder (see this to find out where the profile folder is) and see this example user.js file that you can modify. Created by techlifeweb.com, this example explains some of the things you can do in its comments.

10) about:config. The true power user’s tool, about.config isn’t something to mess with if you don’t know what a setting does. You can get to the main configuration screen by putting about:config in the browser’s address bar. See Mozillazine’s about:config tips and screenshots.

11) Add a keyword for a bookmark. Go to your bookmarks much faster by giving them keywords. Right-click the bookmark and then select Properties. Put a short keyword in the keyword field, save it, and now you can type that keyword in the address bar and it will go to that bookmark.

12) Speed up Firefox. If you have a broadband connection (and most of us do), you can use pipelining to speed up your page loads. This allows Firefox to load multiple things on a page at once, instead of one at a time (by default, it’s optimized for dialup connections). Here’s how:

Type “about:config” into the address bar and hit return. Type “network.http” in the filter field, and change the following settings (double-click on them to change them):

Set “network.http.pipelining” to “true”

Set “network.http.proxy.pipelining” to “true”

Set “network.http.pipelining.maxrequests” to a number like 30. This will allow it to make 30 requests at once.

Also, right-click anywhere and select New-> Integer. Name it “nglayout.initialpaint.delay” and set its value to “0?. This value is the amount of time the browser waits before it acts on information it receives.

13) Limit RAM usage. If Firefox takes up too much memory on your computer, you can limit the amount of RAM it is allowed to us. Again, go to about:config, filter “browser.cache” and select “browser.cache.disk.capacity”. It’s set to 50000, but you can lower it, depending on how much memory you have. Try 15000 if you have between 512MB and 1GB ram.

14) Reduce RAM usage further for when Firefox is minimized. This setting will move Firefox to your hard drive when you minimize it, taking up much less memory. And there is no noticeable difference in speed when you restore Firefox, so it’s definitely worth a go. Again, go to about:config, right-click anywhere and select New-> Boolean. Name it “config.trim_on_minimize” and set it to TRUE. You have to restart Firefox for these settings to take effect.

15) Move or remove the close tab button. Do you accidentally click on the close button of Firefox’s tabs? You can move them or remove them, again through about:config. Edit the preference for “browser.tabs.closeButtons”. Here are the meanings of each value:

0: Display a close button on the active tab only

1:(Default) Display close buttons on all tabs

2:Don’t display any close buttons

3:Display a single close button at the end of the tab bar (Firefox 1.x behavior)