Archive for the 'Lasso' Category

Lasso Developer Conference 2008

Over the past few weeks, I’ve been preparing for this year’s Lasso Developer Conference in Chicago. It’s my second time speaking at the event, and although it’s not until September, a written paper and demo materials which will accompany the presentation were due July 1st are due July 14th (thanks to LassoSoft for the gracious extension). I’m really looking forward to the event. It’s truly a great chance to meet and interact with members of the Lasso community as well as the LassoSoft team.

The title of my talk is “Server Side Techniques for Client Side Optimization”, and it explores ways to streamline the delivery of modern web apps by reducing HTTP requests, combining, minifying, and compressing assets with tools such as the YUI Compressor, using asset subdomains, and more, all wrapped up in a self-contained system that won’t force you to change your existing workflow. And, although I agree somewhat with criticisms that it’s “teaching to the test”, it’ll also help you score an A(100) in YSlow. :P

jQuery File Tree

Cory S.N. LaViska over at A Beautiful Site has created a very nice jQuery plugin for displaying file trees called, naturally, jQuery File Tree. The great thing about it is its simplicity. It doesn’t have an overabundance of bells and whistles, but it’s super easy to integrate with a simple API (it even includes Lasso support), has plenty of hooks to add your own custom functionality, and generates nice clean markup. Check it out!

Using Namespaces To Load Lasso Tags Into Memory

This article describes different ways to load custom tags into memory in Lasso, and presents an alternative method of managing tag libraries.

LassoStartup and LassoLibraries

Lasso provides a number of different ways to make custom tags (and types) available for use in your scripts. Tag definitions can be pasted directly into scripts, or included from external files. Alternatively, tag files can be placed in one of two special folders within the Lasso application folder, LassoStartup and LassoLibraries, for automatic loading.

Placing tags within these special folders provides two distinct advantages. First, Lasso handles the loading of the tags for you, so it’s not necessary to manually include any extra files or code in your scripts; the tags are simply always available for use. If placed in LassoStartup, they’re loaded as part of the initialization process when the Lasso Service starts. If placed in LassoLibraries (and arranged in a certain way), they’re loaded on-demand, the first time they are used in a script.

Secondly, once loaded, tags placed in these special folders remain resident in memory until the Lasso Service is restarted. This eliminates the overhead of reading a file from disk, parsing the tag definition, constructing an internal prototype, and adding it to the tag registry, a process that must otherwise be repeated for each tag every time a script which uses that tag runs. For a single tag, all of those actions happen in a few milliseconds, but with larger libraries, those milliseconds can add up to noticeable delays.

Despite the advantages, it’s not always convenient or even possible to place tags in these special folders. For instance, if your project is hosted in a shared hosting environment, you may not have access to the Lasso application folder. If you use a version control system such as Subversion, you’d have to maintain separate working copies in order to keep two directories in different locations in sync. Although LassoLibraries provides a mechanism for unloading tags programmatically, reloading custom tags defined in LassoStartup requires a restart of the Lasso Site and/or the entire Lasso Service. Generally speaking, it’s just more convenient to be able to keep all the files related to a single project within a single folder.

Pushing Tags Into The Global Namespace

The ideal setup, then, is one where the developer can both specify the location of tag libraries, and have them loaded into memory. Fortunately, the same mechanism that allows files in LassoLibraries to be loaded into memory on-demand can be used on other paths as well.

You may have noticed that Lasso’s tag names use a categorization system to organize sets of tags into groups. For example, the tags which deal with string manipulation are all prefixed by string_. These groups are called namespaces (namespaces in general | namespaces in Lasso) and they have a number of interesting uses. What’s important to understand in this context, however, is that Lasso’s namespaces are hierarchical, and all descend from a top-level namespace known as the global namespace. Tags defined within the global namespace remain resident in memory, and thanks to the [namespace_using] tag, we can push tag definitions into the global namespace programmatically.

For example, once the following code snippet runs, the [foo] tag will be available for use on any page until Lasso is restarted. Without the namespace tags, [foo] will only exist on the page in which it was defined.

[
	namespace_using(namespace_global);
		define_tag('foo');
			return('bar');
		/define_tag;
	/namespace_using;
]

A Flexible Loader For Managing Tag Libraries

The [tags_load] custom tag uses this technique to “watch” the given folder and make sure all the tags within it are loaded into memory. Basic usage is simply:

[tags_load('/path/to/mytags/')]

Optionally, you can specify what type of files to load by extension (the default is .inc), and a condition for whether or not to refresh all tags:

[
	tags_load(
		'/path/to/mytags/',
		-ext='.lasso',
		-refresh=(var_defined('reloadmytags') && $reloadmytags === true)
	);
]

When the tag loader is called, it will list every file in the given path which matches the given extension. It does not currently recurse into subdirectories, and will ignore files that begin with a period.

It will then iterate through that list, checking to see if a tag by that name already exists. For instance, if the file is “foo_bar.inc”, it will check for a tag called [foo_bar]. If none is found (or the refresh condition has been met), it will include the file using the [library] tag within a [protect] block. Any errors that occur when loading an individual file will be trapped and logged to Lasso’s built-in error database, and the tag loader will move on to the next file. Log entries made by the tag loader are prefixed with “[Tagloader]” like so:

[Tagloader] Error loading [foo_bar]: No permission.

The total number of successfully loaded tags for each path is stored in a global variable. On subsequent calls for that path, if the number of files to load matches the number of tags already loaded (and the refresh condition is not met), the rest of the routine is skipped. Thus, even if the tag loader is called on every page load, it’s usually only doing work if a new tag is added to the path, or a refresh is requested. When it’s finished, the results are logged to the error database:

[Tagloader] 113 of 115 tags loaded successfully.

In its current incarnation, it’s important that each file loaded with the tag loader contain a tag with the same name as the file. Otherwise, that file will be reloaded every time, regardless of whether it’s necessary. So, for instance, if you have all of the tags in the foo_ namespace in a single file called foo_library.inc, you’ll want to include a placeholder tag to register the library with the tag loader. I typically just define a simple tag which returns “Loaded” and place it at the end of the file (so that it only loads if there are no other errors):

[
	define_tag('foo_bar');
		...
	/define_tag;

	...

	define_tag('foo_library');
		return('Loaded');
	/define_tag;
]

The tag loader could be extended with more options for greater flexibility, but the current feature set has met my needs well for quite some time. Some ideas include:

  • Optionally accepting an array instead of a string for the path, so that multiple locations can be loaded via a single tag call.
  • Recursing subdirectories so that tags can be broken down into nested folders.
  • Replacing the -ext param with -match and/or -deny params which would accept arrays of conditions to match against filenames.
  • An additional -usenamespace param which could override or append to the default global namespace.

Hopefully a future version of Lasso will support custom paths to LassoLibraries and/or other methods of moving things in and out of memory. The namespace library which ships with Lasso is, itself, written in LassoScript, and could easily be modified to support custom paths. However, that would require maintaining a custom-compiled version of Startup.LassoApp everywhere you want to deploy your code — not a very portable solution. In the meantime, I think this is a satisfactory workaround.

[tags_load] can be downloaded at tagSwap.net

Error Management Techniques for Lasso

An important aspect of web application design is error management. Many factors outside a developer’s control can cause problems, even if the deployed code is free of syntax errors, from changes in the underlying subsystems to edge cases in user input that haven’t been accounted for, to remote services that stop responding. It’s the responsibility of the developer to prepare for the unexpected and attempt to handle whatever errors may crop up over time. The following techniques take advantage of the error management capabilities built into Lasso Professional to trap and log errors, notify system administrators that a problem has occurred, and provide a friendly message to end users.

Set the Error Reporting Level to Full

There are three levels of error reporting available for Lasso: Full, Minimal, and None. The error reporting level can be set globally from the administration interface, and/or adjusted at the page level through the use of the [lasso_errorreporting] tag.

Lasso developers are often advised to set the error reporting level to Minimal or None in production in order to reduce the amount of debugging information shown to site visitors when an error occurs, and for good reason. Legitimate users have no use for the debugging data, and the less information you divulge to would-be hackers about the architecture of your system, the better.

So why do I recommend setting the error reporting level to Full? Simple. We’re going to use a custom error page anyway (see below) and we’ll control the display of debugging information there. Also, since we want to log as much information as possible about the error, we’ll want the contents of [error_msg] to be as detailed as we can make it. The only way to get [error_msg] to return a full stack trace for an error is to set the error reporting level to Full.

If your application follows a “onefile” type of methodology where you have a master controller file and/or a global configuration file that’s loaded as part of any and all requests, I would recommend leaving the server-wide setting at Minimal or None, and overriding it to Full from within your application. That way you don’t have to worry about visitors seeing errors created by other apps on the same server, such as the built-in Admin apps, or some utility script you may have forgotten about.

Merge File Errors with Regular Errors

One of the more peculiar idiosyncracies of Lasso is the fact that errors thrown by file manipulation tags are handled separately from all other types of errors. File-specific errors won’t be reported by [error_code] or [error_msg] tags (or any of their brethren in the error_ namespace), nor will they trigger [handle_error] blocks. Thus, file errors won’t cause an error page to be displayed. Instead, you must check [file_currenterror] when working with the file tags. It’s easy to overlook this separate error tag, which can lead to confusion when debugging.

Fortunately, we can alter this behavior so that file errors are reported as regular errors. In the following code snippet, a [handle] block is defined that checks for the existence of a file error. If a file error is detected*, a [fail] tag is used to trigger a normal error, and the normal error code and error message are set to match the code and message that [file_currenterror] reports.

handle(file_currenterror( -errorcode));
    fail(
        file_currenterror( -errorcode),
        file_currenterror
    );
/handle;

* Here’s how this works: [file_currenterror( -errorcode)] will return just the current error number. If there’s no error, that number will be zero. Zero evaluates to false. All other numbers evaluate to true. Thus, the handle block will only be triggered if there is an actual file error.

Provide a Custom Error Page

Lasso makes implementing a custom error page so easy there’s no excuse for not having one. By simply placing a file named “error.lasso” at the root level of your web site, you can replace the default “blue screen” error page with anything you like. It’s a good idea to match the look and feel of the rest of your application, but otherwise keep the page as simple as possible. Attempting to use a lot of complex logic within a custom error page can quickly lead to problems, since errors in your error management code can create a recursive failure (even though Lasso will bail out of the loop when the recursion limit is reached). Nonetheless, there are a few simple things worth doing, which are outlined below.

Capture the Error Code and Message Right Away

Since we are going to be doing a bit of code within our custom error.lasso page, let’s make sure that we don’t accidentally reset the error code or message that we’ll be working against:

var(
    'code' = error_code,
    'msg' = error_msg
);

Inline Errors vs. Full Page Errors

If an error occurs within an [include], Lasso will display the contents of error.lasso inline with the rest of the calling page, whereas otherwise the entire calling page will be replaced with the contents of error.lasso. This can be problematic if your custom error.lasso page uses the same template as the rest of your site, since inline errors will display the entire template twice: once for the calling page and once for the area where an [include] threw an error.

Ideally, we want to detect whether a given error came from within an include or from the calling page itself, so that we can alter the presentation of the error message accordingly. We do this by inspecting the error stack (which, again, requires that the error reporting level be set to Full):

var('errorInline') = ($msg >> 'at: include with params:');

If the error stack references [include], we know that the error occurred within an included file and that our error.lasso page will be displayed within the context of the calling page. Now we can easily enclose portions of our error management code in conditional statements for more precise control:

if($errorInline);
    // only do this for inline errors
/if;

if(!$errorInline);
    // only do this for full-page errors
/if;

Display a Friendly Message to Users

Visitors should be presented with a simple message explaining that the operators of the site have been notified of a potential problem. There’s no need for them to see any technical details about the error itself. An example message might look something like this:

An error has occurred. The error has been logged and the system administrator has been notified. You may go back and try again now, or, if the error persists, try again later. We apologize for the inconvenience.

Return An Appropriate HTTP Status Code

In addition to the message you provide for the human user, you should also deliver the correct message to the user agent (usually a web browser, but also search engine spiders and other programs) in the form of an HTTP status code. By default, Lasso will return status code 500 for most errors, and status code 401 for some permission-based errors. These are perfectly reasonable defaults, but you may wish to be more specific. A few potential matches are listed below:

Lasso Errors:
-9963 Invalid password. The password supplied is not valid.
-9964 Invalid user name. The user name supplied is not valid.

HTTP Status Code:
401 Unauthorized
The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in “HTTP Authentication: Basic and Digest Access Authentication”.
Lasso Error:
-9961 No permission. The current user is not allowed to perform the specified action. This could mean that a file suffix is not allowed by Lasso security. Edit user security permissions as configured within Lasso security.

HTTP Status Code:
403 Forbidden
The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.
Lasso Errors:
-9967 Resource not found.
-9984 Unauthorized file suffix or file not found. The error -9984 can be seen if you specify a Lasso page with a file suffix which is not included in the Lasso Security settings. Also returned by file management tags.

HTTP Status Code:
404 Not Found
The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.

A full list of HTTP 1.1 status codes and their definitions can be found here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

To set the HTTP status code according to the list above, you could use code like this within your custom error.lasso page:

if(!$errorInline);                                    // only for full-page
    var('statuses') = map(
              0    =    '200 Okay',                            // just in case
        -9963    =    '401 Unauthorized',
        -9964    =    '401 Unauthorized',
        -9961    =    '403 Forbidden',
        -9967    =    '404 Not Found',
        -9984    =    '404 Not Found'
    );

    var('status') = $statuses->find($code);
    !$status ? $status = '500 Internal Server Error';    // default to 500

    $__http_header__ = string_replaceregexp(
        $__http_header__,
        -find='(^HTTP\\S+)\\s+.*?\r\n',
        -replace='\\1 ' + $status + '\r\n'
    );
/if;

Log As Much Information About The Error As Possible

Once an error has been identified, the application should log as much detail as possible about the event to aid in subsequent debugging efforts. The simplest way to accomplish this is via the [log_critical] tag, which will insert whatever message you provide into Lasso’s built-in error database, timestamped and flagged as critical. (BTW, the default error.lasso page will log the error code and message to the error database at the “detail” level if the error reporting level is less than Full.)

While extremely convenient, there are some downsides to relying on the built-in error database. For one, longer error messages will be truncated due to the column size used in the errors table, although I assume it’s possible to alter the column size using the database browser utility from within SiteAdmin. Also, the UI for viewing logged messages doesn’t provide any way to sort or filter entries beyond the static list of error levels: critical, warning, detail, and deprecated. As a result, finding a particular series of errors, especially when more than one virtual host is handled by the same Lasso Site, can be a very tedious process.

Nevertheless, we get this functionality “for free”, and it’s a good idea to make use of it. I prefer to prefix my log entries with the complete URL the user submitted when the error occurred. This is a succinct way of separating errors between different virtual hosts, and often pinpoints the exact conditions required (i.e., a particular set of GET arguments in the query string) to reproduce the error. So, my log entry looks like this:

// log error to error database
var('desc') = ('[' + client_url + '] ' + error_code + ': ' + error_msg);
log_critical($desc);

However, for the reasons outlined above, I also log additional information to my own custom audit tables, where I can include much more information that I can search and filter as needed. Whether you log to a database or a file, some of the things you should consider logging include:

Information about the user.
If your application requires a login, and the user is authenticated, log the user’s ID. You may also want to log another unique identifier that is easier to use when filtering messages, such as a username or email address. If you use Lasso’s built-in session manager, you may also want to log the user’s session ID (using the [session_id] tag), in case there’s a clue in the sessions table.

Information about the request.
Log the complete request header sent by the user agent using [client_headers]. I also prefer to break out the client’s IP using [client_address], and, as mentioned above, the complete URL requested using [client_url]. You could also log the user agent separately (Bil Corry’s [lp_client_browser] tag is especially useful for this), but the raw form of that information is already included in [client_headers], and since we’re logging server-side errors, it’s usually of little value in debugging.

Send A Notification

In addition to logging, the application should immediately notify the developer when a problem occurs. The simplest solution is to use [email_send] to fire off an email containing the same information you chose to log. Other possibilities include gateways to other realtime messaging systems like SMS, IM, or even Twitter — anything you tend to check on a regular basis. It may seem like a nuisance if a given site starts to generate a lot of errors, but remember that it’s also a nuisance to your clients, customers, and end users. Of course, you’ll want to disable notifications when in a development environment.

Further Reading

LassoSoft has published several Tip of the Week articles on the subject of error management:

The pageblocks framework includes integrated error management:

Smart String Truncation in Lasso

I needed to generate some short intros/teasers from longer body text today, and not having anything readily at hand, decided to see what was available at tagSwap. Searching for “truncate” brought up two tags, one based on the other.

The first is [string_truncate] by John Burwell. It’s pretty straightforward. Pass in a string and the desired length, and it will return the string truncated to exactly the specified length, plus an optional string to indicate the continuation (for instance, an ellipsis). It checks to see if the source string is shorter than the given length, in which case it returns the string unaltered.

This looked like it would do the trick, but I decided to check out the other tag anyway. It was [gf_lowtext] by Gaetano Frascolla. Gaetano’s tag is based on John’s, but adds an additional check to see if the truncated string ends with a space. If not, it assumes a word is being split, and truncates an additional character until it reaches a space before returning the result.

It’s a nice enhancement, but only checking for a space presents some potential limitations. First of all, there are plenty of other whitespace characters that may break up words, and secondly, it doesn’t take punctuation and other special characters into consideration. I’d rather not see the result end with a comma or apostrophe, especially when followed by an ellipsis.

It seemed as if a little regex and some elbow grease might provide me with a few additional enhancements, so I grabbed Gaetano’s tag and got to work. The result is below:

define_tag(
	'truncate',
	-namespace='string_',
	-req='text',
	-req='length', -type='integer', -copy,
	-priority='replace',
	-encodenone,
	-description='Truncates the given string to the given number of characters.'
);
	// if the original string is shorter than or equal to the desired length,
	// just return it unaltered.
	#text->size <= #length ? return(#text);

	local('out') = string;

	// while #out is empty, #length is still greater than zero,
	// and the last character of the new string is not whitespace...
	while(!#out->size || !#out->iswhitespace(#out->size) && #length);
		// store a new substring in #out
		#out = #text->substring(1, #length);
		// decrement #length by 1
		#length -= 1;
	/while;

	// if we reached zero, return nothing
	!#length ? return;

	// remove any trailing non-alphanumeric characters and whitespace
	#out = string_replaceregexp(
		#out,
		-find='[^A-Za-z0-9]*\\s*$',
		-replace=''
	);

	// return the final result with an ellipsis character appended
	return(#out + '…');
/define_tag;

The changes I made include:

  • Using [string->iswhitespace] to check for any whitespace character (tabs, newlines, etc.) instead of just spaces.
  • Returning null if there is no reasonable place to truncate the string within the desired length. This may be an unlikely edge case in normal usage, but without the additional check for #length in the [while], there is the potential to create an endless loop. (For instance, in my test code below.)
  • Trimming not only the whitespace from the result, but also any non-alphanumeric characters. This took care of the “hanging punctuation” issue and seemed reasonable for English strings. Additional exceptions could be added for accented characters.
  • Appending an HTML-encoded ellipsis character to the result automatically instead of a user supplied value and/or three periods. This is the only way I’ve ever wanted to show the continuation, so I didn’t bother making it optional.

To test the tag, I looped the length of a test string to see where it would break given every possible position:

var('str') = 'The quick, brown-fox jumps over the "lazy" dog.';

loop($str->size);
    loop_count + ' - ' + string_truncate($str, loop_count) + '\n';
/loop;

…resulting in the following output:

1 -
2 -
3 -
4 - The…
5 - The…
6 - The…
7 - The…
8 - The…
9 - The…
10 - The…
11 - The quick…
12 - The quick…
13 - The quick…
14 - The quick…
15 - The quick…
16 - The quick…
17 - The quick…
18 - The quick…
19 - The quick…
20 - The quick…
21 - The quick, brown-fox…
22 - The quick, brown-fox…
23 - The quick, brown-fox…
24 - The quick, brown-fox…
25 - The quick, brown-fox…
26 - The quick, brown-fox…
27 - The quick, brown-fox jumps…
28 - The quick, brown-fox jumps…
29 - The quick, brown-fox jumps…
30 - The quick, brown-fox jumps…
31 - The quick, brown-fox jumps…
32 - The quick, brown-fox jumps over…
33 - The quick, brown-fox jumps over…
34 - The quick, brown-fox jumps over…
35 - The quick, brown-fox jumps over…
36 - The quick, brown-fox jumps over the…
37 - The quick, brown-fox jumps over the…
38 - The quick, brown-fox jumps over the…
39 - The quick, brown-fox jumps over the…
40 - The quick, brown-fox jumps over the…
41 - The quick, brown-fox jumps over the…
42 - The quick, brown-fox jumps over the…
43 - The quick, brown-fox jumps over the “lazy…
44 - The quick, brown-fox jumps over the “lazy…
45 - The quick, brown-fox jumps over the “lazy…
46 - The quick, brown-fox jumps over the “lazy…
47 - The quick, brown-fox jumps over the “lazy” dog.

I’m satisfied with the results so far, but of course suggestions are welcome.