Automatically Load New Rows When Reaching Page End

The normal way to handle long lists and tables are to implement paging, paging is very useful and practical when it comes to handling a lot of tabular data. But it might not always be the most intuitive way to handle lists and timelines. This is where dynamic row fetching comes to play and in this article I will show you how to implement it in ASP.NET.

Dynamic row fetching is quite simple and works like this; When you first enter the page it automatically loads the first, for instance, 20 rows, this could be the equivalent of page one in a regular table. When the user had read the rows and scrolled to the bottom of the page another 10 rows are loaded and appended to the bottom of the list making it longer, this is the equivalent page 2. And when the user reaches the bottom again another 10 rows are loaded and so on until all rows has been loaded.

If you don’t follow my explanation, just head over to your favorite twitter feed, for instance @World_Wide_What, and scroll to the bottom of the page and you’ll see what I mean.

There is one issue with this technology though, you can’t have anything else below the the list if you want to keep usability. Cause if your visitor tries to read information below the list he or she needs to scroll to the bottom of the page which will cause the list to load new rows. This will result in that the information your visitor is trying to read gets pushed down and outside the browser window making it impossible or at least hard to read.

Well, enough with all the boring mumbo jumbo, let’s do some coding.

Step 1. The Server Side Code

In this example I will use my twitter timeline as the data source for my data, but you can use whatever you like, such as a SQL database or RSS feed.

Create a new project in Visual Studio of the type ASP.NET Empty Web Application. Add a new Web Form to your project named Default.aspx and a class named Response.cs.

The Response.cs will contain two classes, Tweet which will contain the data related to every tweet, such as Author, Created and Message and the Response class which will contain a List containing every tweet within the range we’d return to our page. It will also contain ReachedEnd which is a boolean telling us if all tweets has been loaded.

public class Response
{
    /// <summary>
    /// True if all tweets has been loaded
    /// </summary>
    public bool ReachedEnd { get; set; }

    /// <summary>
    /// The tweets loaded
    /// </summary>
    public List<Tweet> Tweets { get; set; }
}

public class Tweet
{
    /// <summary>
    /// Date and time tweet was posted
    /// </summary>
    public DateTime Created { get; set; }

    /// <summary>
    /// The author of the tweet
    /// </summary>
    public string Author { get; set; }

    /// <summary>
    /// The tweet itself
    /// </summary>
    public string Message { get; set; }
}

Now that we have the class that will contain the response it’s time to create our web method. The following code goes into your Default.aspx.cs file.

/// <summary>
/// Gets the tweets in the selected range from the server
/// </summary>
[WebMethod]
public static object GetPosts(int skip, int take)
{
    List<Tweet> tweets = Twitter.GetTweets();

    var toDisplay = from t in tweets.Skip(skip).Take(take)
                    select t;

    return new Response {
        ReachedEnd = tweets.Count <= skip + take,
        Tweets = toDisplay.ToList<Tweet>()
    };
}

First we need to get all the tweets from our twitter timeline, this can be replaced with whatever data source you like, such as you SQL Server. I won’t explain this function in detail as it isn’t a part of the tutorial but you have the code below.

Next we’ll use a Linq query filter out the tweets we’d like to return. The skip parameter tells Linq how many tweets we want to skip before getting the tweets we want to show and the take is how many we would like to return. These parameter are sent from the JavaScript function which we’re going to create later on.

Then we create a new instance of the Response class which we created earlier and if the total number of tweets in our timeline is less or equal to the sum of the skip and take we’ve reached the end of our timeline. Otherwise there’s more rows to show when the user scrolls further down. The Tweets property is set to the result of our Linq question.

This is the code for the GetTweets() function which is used to fetch tweets from your favorite timeline. I thought it was nice to share the code but won’t explain this code for your though, since it isn’t a part of the tutorial.

private const string twitterUrl = "http://api.twitter.com/1/statuses/user_timeline.xml?screen_name=World_Wide_What&count=1000&include_rts=true";

public static List<Tweet> GetTweets()
{
    HttpWebRequest twitterRequest = (HttpWebRequest)WebRequest.Create(twitterUrl);
    List<Tweet> tweets = new List<Tweet>();

    XDocument twitterResponse = XDocument.Load(twitterRequest.GetResponse().GetResponseStream());

    foreach (XElement tweet in twitterResponse.Descendants("status"))
    {
        tweets.Add(new Tweet
        {
            Author = tweet.Elements("user").Elements("screen_name").First().Value,
            Created = DateTime.ParseExact(tweet.Element("created_at").Value, "ddd MMM dd HH:mm:ss zzz yyyy", CultureInfo.InvariantCulture),
            Message = tweet.Element("text").Value
        });
    }

    return tweets;
}

That’s it, nothing else is required on the server side, simple and clean.

Step 2. The HTML

Now it’s time to create our client side code, open up your default.aspx and write the following HTML.

<!DOCTYPE html>

<html>
    <head>
        <title>Dynamic Row Fetcher</title>

        <script src="http://ajax.aspnetcdn.com/ajax/jQuery/jquery-1.7.1.min.js">
        </script>
        <script>
        </script>
    </head>
    <body>
        <form id="frmMain" runat="server">
        <div id="main">
            <h1>Dynamic Row Fetcher</h1>
            <p>Showing twitter posts from @World_Wide_What</p>
            <div id="feed">
            </div>
            <div id="loading">
                Loading tweets...
            </div>
        </div>
        </form>
    </body>
</html>

Most of it is quite self explanatory, There’s no stylesheet in the example above so it will look awful but you can download it together with the rest of the code by clicking the link in the bottom of the article.

We’ll use jQuery to make things easier and the script tag will contain all the JavaScript we need to write for this to work. The feed div will be the container for our posts and the loading will notify the user that we’re currently loading tweets.

Step 3. Our JavaScript

Next it’s time to create our JavaScript, all the code below will go into the empty script tag we created in the previous step.

var isLoading = false;
var skip = 0, take = 20;

function loadTweets() {
    isLoading = true;

    $.ajax({
        type: 'POST',
        url: "Default.aspx/GetPosts",
        data: '{ "skip" : ' + skip + ', "take" : ' + take + ' }',
        contentType: "application/json; charset=utf-8",
        dataType: "json",
        success: function (msg) {
            for (var i = 0; i < msg.d.Tweets.length; i++) {
                appendTweet(msg.d.Tweets[i]);
            }

            if (msg.d.ReachedEnd) {
                $('#loading').html('<a href="#main">Back to top</a>');
            }
            else {
                skip += msg.d.Tweets.length;
                take = 10;
                isLoading = false;
            }
        }
    });
}

This is the loadTweets() function, it will make a request to the server using AJAX and handle the response. The for loop loops through the tweets returned in our list inside the Response class created earlier in the code behind. It calls the appendTweet to add them to our page, which we’ll create later on.

Next we check the ReachedEnd boolean to see if there’s more tweets to load or if everything is loaded. If everything is loaded we replace the “Loading…” text with a link for the user to go back to the top of the page. We’ll keep the isLoading as false to prevent our AJAX function from trying to load new rows.

If we haven’t reached the end we’ll increase the skip to the number of tweets loaded so we won’t load the same tweets twice. We also change the take to 10, this will result in that where loading 20 tweets on page load and then another 10 each time the user scrolls to the bottom of the page. Last but not least we set isLoading to false to allow new tweets to be loaded.

For our tweets to show up on the page we need to create the appendTweet function, it will look like this.

function appendTweet(data) {
    var tweet = $('<div>');

    var header = $('<h2>');
    header.html(data.Author + ' <span class="light">' + formatDate(data.Created) + '</span>');

    var message = $('<p>');
    message.html(data.Message);

    tweet.append(header);
    tweet.append(message);

    $('#feed').append(tweet);
}

function formatDate(jsonDate) {
    var parsedDate = new Date(parseInt(jsonDate.substr(6)));
    var monthNames = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];

    return parsedDate.getDate() + ' ' + monthNames[parsedDate.getMonth()];
}

First we create the container div, then we create the header which will contain the author and date the tweet was posted. Since JSON dates are formatted as “/Date(1224043200000)/” which isn’t very user friendly we need to write a small helper function to format it properly.

Next we create a paragraph and appends the message to it then we take our newly created header and message and appends it to the tweet which we’ll append to the bottom of our feed.

The formatDate takes our JSON date and converts it to a JavaScript date object which we use to get the month and date and then we show in the format “dd MMM”, “5 Jan” for instance.

Almost there, the last bit is load the 20 first tweets and append an event that fires when the user scrolls the page.

$(document).ready(function () {
    loadTweets();

    $(window).scroll(function () {
        // Start loading when 200px from the bottom
        if ($(window).scrollTop() + $(window).height() > $('#main').height() - 200 && !isLoading) {
            loadTweets();
        }
    });
});

When the document has finished loading the ready event will be fired, this calls the loadTweets function to load the first page.

Next we’ll append a scroll even handler on the browsers window. Inside the event we add a condition to check if the user is 200px from the bottom of the feed and if we’re not currently loading new posts. If that is true we’ll call the loadTweets function to load new tweets.

Step 4. Try it out

Now were finished, it’s time to try it out build and execute the code the check the result. You can also try my example below if you’re lazy and just skipped to the end. You will also find a download link for the complete example below.