Pagination & Infinite Scroll

Essential techniques for efficiently loading and displaying large datasets in mobile applications.

Quick Decision

Situation	Approach	Watch Out
Simple small list	Offset pagination	Slows on large data
Large changing list	Cursor pagination	Ordering must be stable
Social/feed experience	Infinite scroll	Refresh and scroll restore are needed
Controlled navigation	Paged UI	Can feel less fluid on mobile

Production Checklist

• Problem: At which data volume and change rate will the list operate?
• Solution: Are cursor, page size, prefetch, cache, empty/error state, and refresh behavior clear?
• Trade-off: Infinite scroll feels fluid; position restore and memory management get harder.
• Failure mode: Duplicate items, missing items, stale cursors, endless spinners, and memory growth should be handled.
• Measurement: Track page latency, loaded item count, duplicate rate, scroll jank, and data usage.
• Security/cost: Cursors must not create guessable authorization gaps; large page payloads are expensive.

Pagination Architecture Patterns

Traditional Pagination

• Offset-Based Pagination:
  • Page-based navigation with ?page=2&limit=20
  • Simple implementation but performance issues with large datasets
  • Database query optimization challenges with skip-limit pattern
• Cursor-Based Pagination:
  • Stateful navigation with ?after=cursor_id&limit=20
  • Consistent results even with data changes
  • Better performance for large datasets

Mobile-Optimized Pagination Strategies

• Progressive Loading: Small initial page, larger subsequent pages
• Adaptive Page Sizes: Dynamic page sizing based on network conditions
• Predictive Loading: Intelligent prefetching based on user behavior patterns

Platform-Specific Pagination Implementations

Android Pagination Solutions

• Paging 3 Library Architecture:
  • Data loading abstraction with PagingSource
  • Network + database coordination with RemoteMediator
  • RecyclerView integration with PagingDataAdapter
  • Loading/error state handling with LoadState
• Advanced Features:
  • Grouped content with Separators
  • Additional content with Headers/Footers
  • Error recovery with Retry mechanisms
  • Smooth scrolling with Placeholder support

// Android Paging 3 Implementation
class ItemPagingSource(
    private val apiService: ApiService,
    private val query: String
) : PagingSource<Int, Item>() {
    
    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Item> {
        return try {
            val page = params.key ?: 1
            val response = apiService.getItems(
                query = query,
                page = page,
                size = params.loadSize
            )
            
            LoadResult.Page(
                data = response.items,
                prevKey = if (page == 1) null else page - 1,
                nextKey = if (response.items.isEmpty()) null else page + 1
            )
        } catch (exception: Exception) {
            LoadResult.Error(exception)
        }
    }
    
    override fun getRefreshKey(state: PagingState<Int, Item>): Int? {
        return state.anchorPosition?.let { anchorPosition ->
            state.closestPageToPosition(anchorPosition)?.prevKey?.plus(1)
                ?: state.closestPageToPosition(anchorPosition)?.nextKey?.minus(1)
        }
    }
}

// ViewModel Integration
class ItemViewModel(private val repository: ItemRepository) : ViewModel() {
    val items = Pager(
        config = PagingConfig(
            pageSize = 20,
            enablePlaceholders = false,
            prefetchDistance = 5
        ),
        pagingSourceFactory = { ItemPagingSource(repository.apiService, searchQuery) }
    ).flow.cachedIn(viewModelScope)
}

// Fragment Usage
class ItemFragment : Fragment() {
    private val adapter = ItemPagingAdapter()
    
    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        
        recyclerView.adapter = adapter
        
        viewModel.items.observe(this) { pagingData ->
            adapter.submitData(lifecycle, pagingData)
        }
        
        // Load state handling
        adapter.addLoadStateListener { loadState ->
            when (loadState.refresh) {
                is LoadState.Loading -> showProgressBar()
                is LoadState.Error -> showError(loadState.refresh.error)
                is LoadState.NotLoading -> hideProgressBar()
            }
        }
    }
}

iOS Pagination Solutions

• UITableView/UICollectionView Prefetching:
  • Prefetch data source delegate for predictive loading
  • Cell prefetching for smooth scrolling
  • Background loading coordination
• Combine Framework Integration:
  • Publisher-based pagination flows
  • Automatic retry and error handling
  • State management with CurrentValueSubject

// iOS Pagination Manager
class PaginationManager<T: Codable> {
    private var currentPage = 1
    private var isLoading = false
    private var hasMorePages = true
    
    @Published var items: [T] = []
    @Published var loadingState: LoadingState = .idle
    
    private let pageSize: Int
    private let apiService: APIService
    private var cancellables = Set<AnyCancellable>()
    
    init(pageSize: Int = 20, apiService: APIService) {
        self.pageSize = pageSize
        self.apiService = apiService
    }
    
    func loadNextPage() {
        guard !isLoading && hasMorePages else { return }
        
        isLoading = true
        loadingState = .loading
        
        apiService.fetchItems(page: currentPage, limit: pageSize)
            .receive(on: DispatchQueue.main)
            .sink(
                receiveCompletion: { [weak self] completion in
                    self?.isLoading = false
                    switch completion {
                    case .failure(let error):
                        self?.loadingState = .error(error)
                    case .finished:
                        self?.loadingState = .idle
                    }
                },
                receiveValue: { [weak self] response in
                    guard let self = self else { return }
                    
                    self.items.append(contentsOf: response.items)
                    self.currentPage += 1
                    self.hasMorePages = response.items.count == self.pageSize
                }
            )
            .store(in: &cancellables)
    }
    
    func refresh() {
        currentPage = 1
        hasMorePages = true
        items.removeAll()
        loadNextPage()
    }
}

// SwiftUI Integration
struct InfiniteScrollView<T: Codable & Identifiable>: View {
    @StateObject private var paginationManager: PaginationManager<T>
    
    var body: some View {
        List {
            ForEach(paginationManager.items) { item in
                ItemRow(item: item)
                    .onAppear {
                        if item.id == paginationManager.items.last?.id {
                            paginationManager.loadNextPage()
                        }
                    }
            }
            
            if paginationManager.loadingState == .loading {
                ProgressView()
                    .frame(maxWidth: .infinity, alignment: .center)
            }
        }
        .refreshable {
            paginationManager.refresh()
        }
        .onAppear {
            if paginationManager.items.isEmpty {
                paginationManager.loadNextPage()
            }
        }
    }
}

Flutter Pagination Solutions

• Built-in Scroll Controllers:
  • ScrollController for scroll position detection
  • NotificationListener for scroll events
  • ListView.builder for efficient item rendering
• External Packages:
  • infinite_scroll_pagination for advanced features
  • pull_to_refresh for refresh functionality
  • cached_network_image for image pagination

// Flutter Pagination Controller
class PaginationController<T> extends ChangeNotifier {
  List<T> _items = [];
  bool _isLoading = false;
  bool _hasReachedMax = false;
  String? _error;
  
  List<T> get items => _items;
  bool get isLoading => _isLoading;
  bool get hasReachedMax => _hasReachedMax;
  String? get error => _error;
  
  final Future<PaginatedResponse<T>> Function(int page) _fetchPage;
  final int pageSize;
  int _currentPage = 1;
  
  PaginationController({
    required Future<PaginatedResponse<T>> Function(int page) fetchPage,
    this.pageSize = 20,
  }) : _fetchPage = fetchPage;
  
  Future<void> loadNextPage() async {
    if (_isLoading || _hasReachedMax) return;
    
    _isLoading = true;
    _error = null;
    notifyListeners();
    
    try {
      final response = await _fetchPage(_currentPage);
      
      if (response.items.isEmpty) {
        _hasReachedMax = true;
      } else {
        _items.addAll(response.items);
        _currentPage++;
        _hasReachedMax = response.items.length < pageSize;
      }
    } catch (e) {
      _error = e.toString();
    } finally {
      _isLoading = false;
      notifyListeners();
    }
  }
  
  Future<void> refresh() async {
    _currentPage = 1;
    _hasReachedMax = false;
    _items.clear();
    await loadNextPage();
  }
}

// Flutter Infinite Scroll Widget
class InfiniteScrollListView<T> extends StatefulWidget {
  final PaginationController<T> controller;
  final Widget Function(BuildContext context, T item) itemBuilder;
  final Widget? loadingWidget;
  final Widget? errorWidget;
  
  const InfiniteScrollListView({
    Key? key,
    required this.controller,
    required this.itemBuilder,
    this.loadingWidget,
    this.errorWidget,
  }) : super(key: key);
  
  @override
  _InfiniteScrollListViewState<T> createState() => _InfiniteScrollListViewState<T>();
}

class _InfiniteScrollListViewState<T> extends State<InfiniteScrollListView<T>> {
  final ScrollController _scrollController = ScrollController();
  
  @override
  void initState() {
    super.initState();
    _scrollController.addListener(_onScroll);
    widget.controller.addListener(_onControllerUpdate);
    
    // Load initial data
    WidgetsBinding.instance.addPostFrameCallback((_) {
      if (widget.controller.items.isEmpty) {
        widget.controller.loadNextPage();
      }
    });
  }
  
  void _onScroll() {
    if (_isBottom) {
      widget.controller.loadNextPage();
    }
  }
  
  bool get _isBottom {
    if (!_scrollController.hasClients) return false;
    final maxScroll = _scrollController.position.maxScrollExtent;
    final currentScroll = _scrollController.offset;
    return currentScroll >= (maxScroll * 0.9);
  }
  
  void _onControllerUpdate() => setState(() {});
  
  @override
  Widget build(BuildContext context) {
    return RefreshIndicator(
      onRefresh: widget.controller.refresh,
      child: ListView.builder(
        controller: _scrollController,
        itemCount: widget.controller.hasReachedMax
            ? widget.controller.items.length
            : widget.controller.items.length + 1,
        itemBuilder: (context, index) {
          if (index >= widget.controller.items.length) {
            return widget.loadingWidget ?? 
                   const Center(child: CircularProgressIndicator());
          }
          
          return widget.itemBuilder(context, widget.controller.items[index]);
        },
      ),
    );
  }
  
  @override
  void dispose() {
    _scrollController.removeListener(_onScroll);
    _scrollController.dispose();
    widget.controller.removeListener(_onControllerUpdate);
    super.dispose();
  }
}

JavaScript/React Native Pagination

• FlatList Optimization:
  • onEndReached for infinite scroll
  • getItemLayout for performance
  • removeClippedSubviews for memory management
• State Management Integration:
  • Redux/Zustand for pagination state
  • React Query for server state management
  • SWR for data fetching and caching

// React Native Pagination Hook
import { useState, useCallback, useRef } from 'react';

export const usePagination = (fetchFunction, pageSize = 20) => {
  const [data, setData] = useState([]);
  const [loading, setLoading] = useState(false);
  const [refreshing, setRefreshing] = useState(false);
  const [hasMore, setHasMore] = useState(true);
  const [error, setError] = useState(null);
  
  const currentPage = useRef(1);
  const isLoadingMore = useRef(false);
  
  const loadMore = useCallback(async () => {
    if (loading || isLoadingMore.current || !hasMore) return;
    
    isLoadingMore.current = true;
    setLoading(true);
    setError(null);
    
    try {
      const response = await fetchFunction({
        page: currentPage.current,
        limit: pageSize,
      });
      
      const newItems = response.data || [];
      
      setData(prevData => 
        currentPage.current === 1 ? newItems : [...prevData, ...newItems]
      );
      
      setHasMore(newItems.length === pageSize);
      currentPage.current += 1;
    } catch (err) {
      setError(err.message);
    } finally {
      setLoading(false);
      isLoadingMore.current = false;
    }
  }, [fetchFunction, pageSize, loading, hasMore]);
  
  const refresh = useCallback(async () => {
    setRefreshing(true);
    currentPage.current = 1;
    setHasMore(true);
    setData([]);
    
    await loadMore();
    setRefreshing(false);
  }, [loadMore]);
  
  const reset = useCallback(() => {
    setData([]);
    setLoading(false);
    setRefreshing(false);
    setHasMore(true);
    setError(null);
    currentPage.current = 1;
  }, []);
  
  return {
    data,
    loading,
    refreshing,
    hasMore,
    error,
    loadMore,
    refresh,
    reset,
  };
};

// FlatList Implementation
import React, { useEffect } from 'react';
import { FlatList, ActivityIndicator, Text } from 'react-native';

const PaginatedList = ({ fetchItems, renderItem, keyExtractor }) => {
  const {
    data,
    loading,
    refreshing,
    hasMore,
    error,
    loadMore,
    refresh,
  } = usePagination(fetchItems);
  
  useEffect(() => {
    loadMore(); // Initial load
  }, []);
  
  const renderFooter = () => {
    if (!hasMore) return null;
    
    return (
      <ActivityIndicator 
        style={{ margin: 15 }} 
        size="small" 
        color="#0066cc" 
      />
    );
  };
  
  const renderError = () => {
    if (!error) return null;
    
    return (
      <Text style={{ textAlign: 'center', color: 'red', margin: 10 }}>
        {error}
      </Text>
    );
  };
  
  return (
    <FlatList
      data={data}
      renderItem={renderItem}
      keyExtractor={keyExtractor}
      onEndReached={loadMore}
      onEndReachedThreshold={0.1}
      onRefresh={refresh}
      refreshing={refreshing}
      ListFooterComponent={renderFooter}
      ListHeaderComponent={renderError}
      removeClippedSubviews={true}
      maxToRenderPerBatch={10}
      updateCellsBatchingPeriod={50}
      initialNumToRender={20}
      windowSize={10}
    />
  );
};

Advanced Pagination Techniques

Cursor-Based Implementation

• Benefits: Consistent pagination even with data changes
• Implementation: Use stable identifiers (timestamps, UUIDs)
• Database Support: Efficient queries with indexed cursor fields

// Cursor-based API endpoint
app.get('/api/posts', async (req, res) => {
  const { after, limit = 20 } = req.query;
  
  const query = {
    ...(after && { createdAt: { $lt: new Date(after) } })
  };
  
  const posts = await Post.find(query)
    .sort({ createdAt: -1 })
    .limit(parseInt(limit) + 1); // +1 to check if there are more
  
  const hasNextPage = posts.length > limit;
  const edges = posts.slice(0, limit);
  
  const pageInfo = {
    hasNextPage,
    endCursor: edges.length > 0 ? edges[edges.length - 1].createdAt : null,
  };
  
  res.json({
    edges: edges.map(post => ({ node: post })),
    pageInfo,
  });
});

Virtual Scrolling for Large Lists

• Use Cases: Lists with thousands of items
• Memory Optimization: Render only visible items
• Platform Libraries: react-window, @tanstack/react-virtual

// React Virtual Scrolling
import { FixedSizeList as List } from 'react-window';
import InfiniteLoader from 'react-window-infinite-loader';

const VirtualizedInfiniteList = ({ 
  items, 
  hasNextPage, 
  isNextPageLoading, 
  loadNextPage 
}) => {
  const itemCount = hasNextPage ? items.length + 1 : items.length;
  const isItemLoaded = index => !!items[index];
  
  const Item = ({ index, style }) => {
    const item = items[index];
    
    if (!item) {
      return (
        <div style={style}>
          <div>Loading...</div>
        </div>
      );
    }
    
    return (
      <div style={style}>
        <ItemComponent item={item} />
      </div>
    );
  };
  
  return (
    <InfiniteLoader
      isItemLoaded={isItemLoaded}
      itemCount={itemCount}
      loadMoreItems={loadNextPage}
    >
      {({ onItemsRendered, ref }) => (
        <List
          ref={ref}
          height={400}
          itemCount={itemCount}
          itemSize={50}
          onItemsRendered={onItemsRendered}
        >
          {Item}
        </List>
      )}
    </InfiniteLoader>
  );
};

Performance Optimization Strategies

Memory Management

• Item Recycling: Reuse view components for better performance
• Image Loading: Lazy loading with placeholder images
• Data Cleanup: Remove off-screen items from memory

Smooth Scrolling Techniques

• Prefetch Distance: Load data before reaching the end
• Throttled Loading: Prevent excessive API calls
• Background Processing: Load data on background threads

Network Optimization

• Request Consolidation: Batch multiple page requests
• Compression: Use gzip compression for API responses
• Caching Strategy: Cache pages for offline browsing

// Advanced Pagination with Caching
class PaginationCache {
  constructor(maxSize = 50) {
    this.cache = new Map();
    this.maxSize = maxSize;
  }
  
  set(key, value) {
    if (this.cache.size >= this.maxSize) {
      const firstKey = this.cache.keys().next().value;
      this.cache.delete(firstKey);
    }
    this.cache.set(key, value);
  }
  
  get(key) {
    const value = this.cache.get(key);
    if (value) {
      // Move to end (LRU)
      this.cache.delete(key);
      this.cache.set(key, value);
    }
    return value;
  }
  
  has(key) {
    return this.cache.has(key);
  }
}

const paginationCache = new PaginationCache();

const fetchPageWithCache = async (page) => {
  const cacheKey = `page_${page}`;
  
  if (paginationCache.has(cacheKey)) {
    return paginationCache.get(cacheKey);
  }
  
  const response = await api.fetchPage(page);
  paginationCache.set(cacheKey, response);
  
  return response;
};

Best Practices

User Experience Guidelines

• Loading States: Show clear loading indicators
• Error Handling: Provide retry mechanisms for failed loads
• Empty States: Design meaningful empty state screens
• Pull-to-Refresh: Implement intuitive refresh gestures

Performance Considerations

• Page Size Optimization: Balance between load time and user experience
• Debounce Scrolling: Prevent excessive scroll event handling
• Memory Monitoring: Track and optimize memory usage

Accessibility Support

• Screen Reader: Announce loading states and page updates
• Focus Management: Maintain proper focus when new items load
• Keyboard Navigation: Support keyboard-only navigation patterns

This comprehensive pagination system provides efficient data loading while maintaining excellent user experience across all mobile platforms.