My App

Plan

SmartCrew Mobile - Plan

DailyLog Architecture Migration Plan with Caching

Overview

This document outlines the architectural changes required to migrate DailyLog from a user subcollection to a standalone top-level collection with an optimized caching strategy for improved performance and user experience.

Current Architecture Issues

  1. Daily logs stored as subcollection under users collection
  2. Inefficient querying for reporting across multiple users
  3. No caching mechanism for frequently accessed data
  4. Performance issues when navigating between days

Target Architecture

daily_logs (top-level collection)
  ├── documentId: "userId_date" (e.g., "user123_20231025")
  │   ├── userId: string (reference to user document)
  │   ├── companyId: string (reference to company document)
  │   ├── logDate: timestamp
  │   ├── status: string
  │   ├── isDayOff: boolean
  │   ├── dayOffDetails: map
  │   ├── activities: subcollection
  │   └── clockEvents: subcollection

Caching Strategy

Multi-Level Cache Implementation

class DailyLogCache {
  final Map<String, DailyLog> _dailyLogCache = {}; // Key: "userId_date"
  final Map<String, Map<DateTime, DailyLog>> _weeklyLogCache = {}; // Key: "userId_weekStart"
  final Map<String, Map<DateTime, DayLogType>> _statusCache = {}; // Key: "userId_weekStart"
  
  // Cache duration settings
  static const Duration dailyLogCacheDuration = Duration(hours: 1);
  static const Duration weeklyLogCacheDuration = Duration(hours: 6);
  
  // Methods for cache management
  void addToDailyCache(String userId, DateTime date, DailyLog log) {
    final key = _generateDailyKey(userId, date);
    _dailyLogCache[key] = log;
  }
  
  DailyLog? getFromDailyCache(String userId, DateTime date) {
    final key = _generateDailyKey(userId, date);
    return _dailyLogCache[key];
  }
  
  // Similar methods for weekly cache and status cache
  // ...
}

Cache Integration with Provider

The provider will implement a multi-level caching strategy:

  1. Memory Cache: Store recently accessed logs in memory
  2. Local Database: Persist frequently accessed logs using Hive or SQLite
  3. Network: Fetch from Firestore when not available in cache

Required Code Changes

1. Update Data Access Methods

// New method to get daily log with caching
Future<DailyLog> getDailyLogWithCache(String userId, DateTime date) async {
  // Check memory cache first
  final cachedLog = _cache.getFromDailyCache(userId, date);
  if (cachedLog != null) return cachedLog;
  
  // Check local database
  final localLog = await _localDb.getDailyLog(userId, date);
  if (localLog != null) {
    _cache.addToDailyCache(userId, date, localLog);
    return localLog;
  }
  
  // Fetch from Firestore
  final compositeId = DailyLog.generateId(userId, date);
  final doc = await fireStore.collection('daily_logs').doc(compositeId).get();
  
  if (doc.exists) {
    final dailyLog = await _fetchDailyLogWithSubcollections(doc);
    _cache.addToDailyCache(userId, date, dailyLog);
    await _localDb.saveDailyLog(userId, date, dailyLog);
    return dailyLog;
  }
  
  // Return empty log if not found
  return CreateDayLogService.empty(logDate: date);
}

2. Update Weekly Status Methods

Future<void> getStatusPerDayWithCache({required DateTime monday}) async {
  final userId = authProvider.currentUser?.firebaseUid;
  if (userId == null) return;
  
  final cacheKey = '${userId}_${monday.toIso8601String()}';
  
  // Check cache first
  if (_statusCache.containsKey(cacheKey)) {
    _statusPerDay = _statusCache[cacheKey]!;
    notifyListeners();
    return;
  }
  
  // Proceed with network request if not in cache
  final weekLogs = await fireStore.collection('daily_logs')
      .where('userId', isEqualTo: userId)
      .where('logDate', '>=', Timestamp.fromDate(monday))
      .where('logDate', '<=', Timestamp.fromDate(monday.add(Duration(days: 6))))
      .get();
  
  // Process and cache results
  _statusCache[cacheKey] = _processWeekStatus(weekLogs);
  _statusPerDay = _statusCache[cacheKey]!;
  notifyListeners();
}

3. Implement Cache-Aware Write Operations

Future<void> saveDailyLogWithCache(DailyLog log) async {
  // Save to Firestore
  await _saveToFirestore(log);
  
  // Update cache
  _cache.addToDailyCache(log.userId, log.logDate, log);
  
  // Update local database
  await _localDb.saveDailyLog(log.userId, log.logDate, log);
  
  // Invalidate weekly cache for this week
  final monday = getMondayOfCurrentWeek(log.logDate);
  final cacheKey = '${log.userId}_${monday.toIso8601String()}';
  _statusCache.remove(cacheKey);
  _weeklyLogCache.remove(cacheKey);
}

4. Add Cache Management Methods

void clearCache() {
  _dailyLogCache.clear();
  _weeklyLogCache.clear();
  _statusCache.clear();
}

void removeFromCache(String userId, DateTime date) {
  final dailyKey = _generateDailyKey(userId, date);
  _dailyLogCache.remove(dailyKey);
  
  final monday = getMondayOfCurrentWeek(date);
  final weeklyKey = '${userId}_${monday.toIso8601String()}';
  _weeklyLogCache.remove(weeklyKey);
  _statusCache.remove(weeklyKey);
}

Success Criteria

1. Performance Metrics

  • 80% reduction in Firestore read operations for daily log access
  • Sub-100ms response time for switching between days in the same week
  • 50% reduction in data usage for frequent users

2. Functional Requirements

  • All existing features work correctly with new architecture
  • Users can seamlessly navigate between days without loading delays
  • Weekly status widget updates instantly for cached weeks
  • Real-time updates still work for current day

3. Data Consistency

  • Cache is properly invalidated on data updates
  • No stale data shown to users
  • Offline functionality maintained or improved

4. Memory Management

  • Cache uses reasonable memory limits
  • Old cache entries are properly purged
  • No memory leaks in cache implementation

Implementation Checklist

Phase 1: Core Architecture Changes

  • Update DailyLog model with userId field
  • Create composite ID generation utility
  • Update all Firestore references to use new collection
  • Implement data migration script

Phase 2: Caching Implementation

  • Create DailyLogCache class
  • Implement memory caching strategy
  • Add local database persistence (Hive/SQLite)
  • Integrate cache with provider methods

Phase 3: Provider Refactoring

  • Update getDailyLog to use cache
  • Update getStatusPerDay to use cache
  • Implement cache-aware save methods
  • Add cache management methods

Phase 4: Testing

  • Verify cache hit/miss behavior
  • Test cache invalidation on updates
  • Performance testing with large datasets
  • Offline functionality testing

Risk Mitigation

  1. Data Inconsistency

    • Implement proper cache invalidation strategies
    • Use Firestore real-time listeners for critical data
  2. Memory Overhead

    • Implement cache size limits
    • Use efficient data structures for caching
  3. Offline Functionality

    • Ensure local database properly syncs with cache
    • Handle offline writes with queue system
  4. Backward Compatibility

    • Maintain old data structure during transition
    • Implement dual-write during migration period

This plan provides a comprehensive approach to migrating the DailyLog system with an optimized caching strategy that will significantly improve performance and user experience.

On this page