Partial Updates and Retrievals - Summary

Motivation

#

• Problem space

  • Full-resource replace-only APIs cause data loss and concurrency problems (two clients replacing different fields can clobber each other).
  • Large resources or constrained clients (e.g., IoT) need the ability to fetch only specific fields to save bandwidth/memory.

• Two related needs

  • Partial retrievals — request only subset of fields from a resource (important for bandwidth, memory, latency).
  • Partial updates — update only specific fields (important for correctness and avoiding unintended overwrites).

Overview (field mask concept)

#

• A field mask = an array/list of strings (paths) that specify which fields to retrieve or update.

• Use the same concept for both GET (retrieval) and PATCH (update).

  • GET + field mask → return only requested fields.
  • PATCH + field mask → update only requested fields; otherwise leave other fields unchanged.

• If PATCH has no explicit field mask and body is JSON, the service can infer a field mask from the provided attributes (implicit field mask).

• A special sentinel "*" in a field mask means “all fields”.

Example:

GET /chatRoom/1?fieldMask=title
→ returns only id and title (and other mandatory fields as defined)

Implementation details

#

Transporting the field mask

#

• Do not put field mask in the PATCH body (that would break the idea that the body is the resource representation).

• Preferred transport: query parameters (e.g., ?fieldMask=title&fieldMask=description) because:

  • Compatible with GET (no body).
  • Easier to use from browsers.

• Be aware: different frameworks/systems parse multi-value query params differently. Use a consistent library convention (commonly ?fieldMask=a&fieldMask=b).

Field mask path syntax (rules)

#

1. Use . as separator for nested parts (e.g., loggingConfig.maxSizeMb).
2. Use * to refer to all fields of a nested message (e.g., loggingConfig.*).
3. Map keys are strings; if a key cannot be an unquoted identifier, quote it with backticks: settings.`1234` .
4. Use backticks to quote parts that include punctuation (dots, asterisks, numeric-like keys).
5. Escape a literal backtick by doubling: ``` → `````` (i.e., use two backticks for one literal backtick).

Examples (addressing nested maps and odd keys)

#

• description
• loggingConfig.maxSizeMb
• settings.* (all keys in settings map)
• settings.`1234` (numeric-like key)
• settings.`test.value` (map key that contains a dot)
• For literal backticks, double them in the quoted key.

Repeated fields (arrays/lists)

#

• Do not support addressing or updating an item by numeric index (administrators[0]) — index semantics are unstable and imply guaranteed ordering.
• You may select fields across all items using the * wildcard: e.g., administrators.*.name → return every administrator’s name.
• But you cannot safely update individual elements by index via field mask; updating repeated fields requires full replacement of that repeated field (or use maps/sub-resources with stable IDs).

Defaults

#

• GET default: return all fields (unless API documents exceptions).

  • Exception: fields that are extremely large or expensive to compute may be excluded from the default and must be explicitly requested. This must be clearly documented.

• PATCH default: try to infer field mask from the JSON body (i.e., fields present in the body are the intent to update).

• Use "*" sentinel to explicitly request all fields (overrides default exclusions).

Implicit field masks (inference algorithm)

#

• Walk the JSON object; every field present (including null) becomes part of the inferred field mask.
• null is a present value and will be included → explicit set-to-null is allowed.
• Missing fields (not present in JSON) are not included.

Dynamic data structures (maps) and removing keys

#

• JSON has no undefined. To remove a key from a map:

  • Use an explicit field mask that names the map entry to remove and omit that entry from the request body (field is present in mask but absent from body). The server interprets this as “delete that key”.

  • Example:

    PATCH /chatRooms/1?fieldMask=settings.test
    Content-Type: application/json
    { }        ← no settings.test in body → remove settings.test
    

• Do not rely on null to mean “remove” — null is a concrete value and will set the key to null, not remove it.

• In languages like Python, be careful: input.get('settings', {}).get('test') could return None (not the same as undefined); server must detect presence vs absence.

Invalid / unknown fields

#

• When a field listed in a field mask does not exist in the resource or request body:

  • Do not throw an error (avoid brittle clients between versions).
  • Treat missing fields as undefined and proceed (no-op / remove / ignore as appropriate).

• This conservative approach helps tolerate evolving APIs (added/removed fields).

Final API definition (example)

#

• Add a FieldMask type (string array) to request interfaces.
• Example interface definitions:

type FieldMask = string[];

interface GetChatRoomRequest {
  id: string;
  fieldMask?: FieldMask;
}

interface UpdateChatRoomRequest {
  resource: ChatRoom;
  fieldMask?: FieldMask; // optional; if absent, infer from body
}

• Routes:

  • GET /chatRooms/{id}?fieldMask=...
  • PATCH /chatRooms/{id}?fieldMask=...

Trade-offs & guidance

#

• When to support partial retrieval:

  • Not mandatory for every API — only when resources are large, expensive to compute, or clients are constrained.
  • If implemented, prefer to support partial retrieval across the API for consistency (don’t do it on a resource-by-resource whim).

• Field masks are not a query language:

  • They are for selecting fields, not for expressing joins or complex queries. If you need powerful, relational-style queries across many resources, consider GraphQL or another purpose-built query layer.

• Alternatives:

  • JSON Patch (RFC-6902) — powerful, supports operations like test/copy/array-index ops; good if you need operational-level control.
  • JSON Merge Patch (RFC-7396) — similar to inferred field masks, but has quirks (e.g., null semantics, array handling) and is not recursive for repeated fields.

• Design rule: partial updates (PATCH semantics) are a hard requirement for update endpoints — otherwise your PATCH behaves like a replace.

Practical checklist for API designers

#

• Decide whether partial retrieval is needed; if yes, implement it uniformly across resources.
• Add a fieldMask (string[]) parameter to GET and PATCH endpoints (query param).
• Choose and document the query param format for repeated parameters (e.g., &fieldMask=title&fieldMask=description).
• Support the path syntax rules: . separator, * wildcard, backtick quoting and escaping for odd keys.
• Implement implicit field mask inference for PATCH when fieldMask is omitted.
• Treat null as “explicit set to null”; treat absent fields as “no-op” unless field mask indicates deletion.
• For map key deletion: require explicit fieldMask entry and absence of value in body to indicate removal.
• Disallow addressing repeated elements by numeric index; use * to select fields across items, or use maps/sub-resources for stable IDs.
• Do not error on unknown fields in masks — be forward/backward tolerant.
• Document any fields excluded from default GET results (and provide * to request everything).

Short example recap

#

• Retrieve title only:

  GET /chatRooms/1?fieldMask=title
  

• Update title only (implicit field mask):

  PATCH /chatRooms/1
  { "title": "New title" }
  

  server infers fieldMask = [“title”]

• Remove map key settings.test:

  PATCH /chatRooms/1?fieldMask=settings.test
  Content-Type: application/json
  { }   ← settings.test absent → remove it