Add device ping API + UI dialog + TEST_README

git status#

TEST_README.md

@@ -0,0 +1,190 @@
+# ThingsBoard – Device Ping Task Deliverable
+
+This document describes how to build/test/run the changes delivered for the task:
+
+- Add REST API: `GET /api/device/ping/{deviceId}`
+- Add “Ping Device” button on Device Details page (shows popup result)
+- Add unit tests for backend ping logic
+- Provide brief module overview (application/dao/transport/ui)
+
+---
+
+## 1) Build & Test Instructions
+
+### Prereqs
+- Java 17
+- Maven
+- Docker Desktop + Docker Compose
+
+### Backend build
+From repo root:
+```bash
+./build.sh
+```
+
+### Run unit tests (ping tests only)
+```bash
+cd application
+mvn -Dtest=DefaultDevicePingServiceTest test
+```
+
+ ### Running Locally (Docker)
+
+The repo provides scripts and compose overlays under `docker/`.
+A typical working stack for this task used:
+- Postgres
+- Kafka
+- Valkey
+
+### Start (example)
+From `docker/`:
+```bash
+./docker-create-log-folders.sh
+./docker-install-tb.sh --loadDemo
+./docker-start-services.sh
+```
+
+Open:
+- UI: `http://localhost`
+- Login (demo): `tenant@thingsboard.org` / `tenant`
+
+---
+
+## Verifying the Feature (Acceptance Checklist)
+
+### UI check
+1. Open `http://localhost` and login.
+2. Go to **Entities → Devices**.
+3. Open a device and click **Ping Device**.
+4. Confirm a dialog opens showing Online/Offline, Last Seen, and the Device Id.
+
+### API check (curl)
+```bash
+JWT=$(curl -s -X POST http://localhost:80/api/auth/login \
+  -H 'Content-Type: application/json' \
+  -d '{"username":"tenant@thingsboard.org","password":"tenant"}' \
+  | python3 -c 'import sys,json; print(json.load(sys.stdin)["token"])')
+
+DEVICE_ID=$(curl -s -H "X-Authorization: Bearer $JWT" \
+  'http://localhost:80/api/tenant/devices?pageSize=1&page=0' \
+  | python3 -c 'import sys,json; j=json.load(sys.stdin); print(j["data"][0]["id"]["id"])')
+
+curl -s -i -H "X-Authorization: Bearer $JWT" \
+  "http://localhost:80/api/device/ping/$DEVICE_ID"
+```
+Expected: HTTP `200` with JSON including `deviceId`, `reachable`, `lastSeen`.
+
+---
+
+## 2) What Changed (Overview)
+
+### Backend
+- Added new endpoint: `GET /api/device/ping/{deviceId}`
+  - Implemented in `DeviceController` (Spring MVC controller under `/api`).
+  - Computes reachability using existing telemetry:
+    - Prefer `lastActivityTime` key
+    - Fallback to latest timestamp across all latest telemetry keys
+  - Reachability window is configurable (default: **5 minutes**):
+    - `device.ping.reachability.window.minutes` (Spring property)
+- Refactored ping logic into a dedicated service for testability:
+  - `DevicePingService` interface
+  - `DefaultDevicePingService` implementation
+
+### Frontend
+- “Ping Device” button is present on Device Details page.
+- On click, UI calls `GET /api/device/ping/{deviceId}` and shows a dialog with:
+  - Online/Offline status
+  - Last Seen
+  - Device Id
+  - Message
+
+### Tests
+- Added unit tests for ping logic:
+  - `DefaultDevicePingServiceTest` (3 tests: reachable, unreachable, fallback)
+
+---
+
+## 3) Key Challenges Encountered & How They Were Addressed
+
+1. **Wrong endpoint mapping (`/api/api/...`)**
+   - Root cause: class-level `/api` combined with a method path that also started with `/api`.
+   - Fix: controller method mapping uses `/device/ping/{deviceId}` so final route is `/api/device/ping/{deviceId}`.
+
+2. **UI button existed in source but didn’t show in Docker UI**
+   - Root cause: Docker `tb-web-ui` container was serving older bundled UI assets.
+   - Fix: built a local web-ui image that replaces bundled assets with the current `ui-ngx` build output.
+
+3. **Apple Silicon + docker build plugin issues / Docker Hub pull failures**
+   - Root cause: spotify dockerfile-maven-plugin native/JFFI mismatch on ARM, plus intermittent upstream image pull issues.
+   - Fix: used local “install artifacts into existing image” Dockerfile approach for backend and web-ui.
+
+---
+
+## 4) Module Comprehension (Short, per module)
+
+### `application/`
+**Purpose**: Main Spring Boot application layer (REST controllers, security, orchestration).
+
+**Typical flow**:
+UI/Client → `@RestController` endpoint → service layer (business logic) → DAO/Timeseries services → DTO response.
+
+**Ping path**:
+`DeviceController.pingDevice(...)` → `DevicePingService.pingDevice(...)` → Timeseries queries → `DevicePingResponse`.
+
+### `dao/`
+**Purpose**: Persistence/data-access layer.
+
+- Contains repositories/services that read/write entities and telemetry.
+- In this feature, the ping logic relies on existing telemetry access (Timeseries service) exposed via DAO layer.
+
+### `transport/`
+**Purpose**: Device protocol ingress.
+
+- MQTT/HTTP/CoAP/LwM2M/SNMP transports receive device data.
+- Transport normalizes/authenticates incoming messages and forwards into queue/core.
+- Telemetry written by transports is what the ping endpoint uses to infer “reachability”.
+
+### `ui-ngx/` (UI)
+**Purpose**: Angular frontend (Device pages, dialogs, HTTP services).
+
+**Ping UI flow**:
+Device details page → click “Ping Device” → `DeviceService.pingDevice(...)` → popup dialog displays response.
+
+---
+
+## ) Local Docker Image Workarounds (Used to Deploy Updated Code)
+
+These are practical deployment helpers used in this environment.
+
+### Backend (tb-node) local image update
+If your running Docker stack doesn’t include your latest backend code, rebuild server artifacts (`./build.sh`) and then rebuild a local image that installs the produced package onto the existing base image.
+
+The helper Dockerfile used:
+- `msa/tb-node/target/Dockerfile.local`
+
+(Exact command may vary based on your local artifact output.)
+
+### Web UI (tb-web-ui) local image update
+To ensure the Docker-served UI matches your local `ui-ngx` sources:
+
+1. Ensure `ui-ngx` has been built (it happens as part of `./build.sh` in many setups).
+2. Build a local `thingsboard/tb-web-ui:latest` that swaps in the built `public/` assets.
+
+Helper Dockerfile:
+- `msa/web-ui/target/Dockerfile.local`
+
+---
+
+##  Where to Look in the Code
+
+Backend:
+- `application/src/main/java/org/thingsboard/server/controller/DeviceController.java`
+- `application/src/main/java/org/thingsboard/server/service/device/DevicePingService.java`
+- `application/src/main/java/org/thingsboard/server/service/device/DefaultDevicePingService.java`
+- `application/src/test/java/org/thingsboard/server/service/device/DefaultDevicePingServiceTest.java`
+
+Frontend:
+- `ui-ngx/src/app/modules/home/pages/device/device.component.ts`
+- `ui-ngx/src/app/modules/home/pages/device/device.component.html`
+- `ui-ngx/src/app/modules/home/pages/device/device-ping-dialog.component.ts`
+- `ui-ngx/src/app/modules/home/pages/device/device-ping-dialog.component.html`

application/src/main/java/org/thingsboard/server/controller/DeviceController.java

@@ -126,6 +126,11 @@ import static org.thingsboard.server.controller.ControllerConstants.UNIQUIFY_STR
 import static org.thingsboard.server.controller.ControllerConstants.UUID_WIKI_LINK;
 import static org.thingsboard.server.controller.EdgeController.EDGE_ID;
 
+import org.thingsboard.server.common.data.audit.ActionType;
+import org.thingsboard.server.common.data.device.DevicePingResponse;
+import org.thingsboard.server.service.device.DevicePingService;
+
+
 @RestController
 @TbCoreComponent
 @RequestMapping("/api")
@@ -138,6 +143,8 @@ public class DeviceController extends BaseController {
     private final DeviceBulkImportService deviceBulkImportService;
 
     private final TbDeviceService tbDeviceService;
+    
+    private final DevicePingService devicePingService;
 
     @ApiOperation(value = "Get Device (getDeviceById)",
             notes = "Fetch the Device object based on the provided Device Id. " +
@@ -803,4 +810,37 @@ public class DeviceController extends BaseController {
         return deviceBulkImportService.processBulkImport(request, user);
     }
 
+    /**
+     * Ping a device to check its reachability status.
+     *
+     * @param strDeviceId the device ID
+     * @return DevicePingResponse containing reachability status and last seen timestamp
+     */
+    @ApiOperation(value = "Ping Device", notes = "Checks if a device is reachable based on its last activity timestamp")
+    @PreAuthorize("hasAnyAuthority('TENANT_ADMIN', 'CUSTOMER_USER')")
+    @RequestMapping(value = "/device/ping/{deviceId}", method = RequestMethod.GET)
+    @ResponseBody
+    public DevicePingResponse pingDevice(
+            @Parameter(description = "Device ID") @PathVariable("deviceId") String strDeviceId) throws ThingsboardException {
+        checkParameter(DEVICE_ID, strDeviceId);
+
+        try {
+            DeviceId deviceId = new DeviceId(toUUID(strDeviceId));
+
+            // Check device exists and user has access
+            checkDeviceId(deviceId, Operation.READ);
+
+            DevicePingResponse response = devicePingService.pingDevice(getTenantId(), deviceId);
+
+            // Log the action
+            SecurityUser user = getCurrentUser();
+            logEntityActionService.logEntityAction(user.getTenantId(), deviceId,
+                    ActionType.ATTRIBUTES_READ, user, null, "lastActivityTime");
+
+            return response;
+        } catch (Exception e) {
+            throw handleException(e);
+        }
+    }
+
 }

application/src/main/java/org/thingsboard/server/service/device/DefaultDevicePingService.java

@@ -0,0 +1,95 @@
+/**
+ * Copyright © 2016-2025 The Thingsboard Authors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.thingsboard.server.service.device;
+
+import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.stereotype.Service;
+import org.thingsboard.server.common.data.device.DevicePingResponse;
+import org.thingsboard.server.common.data.id.DeviceId;
+import org.thingsboard.server.common.data.id.TenantId;
+import org.thingsboard.server.common.data.kv.TsKvEntry;
+import org.thingsboard.server.dao.timeseries.TimeseriesService;
+
+import java.text.SimpleDateFormat;
+import java.util.Comparator;
+import java.util.Date;
+import java.util.List;
+import java.util.Locale;
+import java.util.concurrent.TimeUnit;
+
+@Service
+@RequiredArgsConstructor
+@Slf4j
+public class DefaultDevicePingService implements DevicePingService {
+
+    private static final String LAST_ACTIVITY_TIME_KEY = "lastActivityTime";
+
+    private final TimeseriesService timeseriesService;
+
+    @Value("${device.ping.reachability.window.minutes:5}")
+    private long reachabilityWindowMinutes;
+
+    @Override
+    public DevicePingResponse pingDevice(TenantId tenantId, DeviceId deviceId) {
+        Long lastActivityTime = getLastActivityTime(tenantId, deviceId);
+
+        long now = System.currentTimeMillis();
+        long windowStart = now - TimeUnit.MINUTES.toMillis(reachabilityWindowMinutes);
+        boolean reachable = lastActivityTime != null && lastActivityTime > windowStart;
+
+        DevicePingResponse response = new DevicePingResponse();
+        response.setDeviceId(deviceId.getId().toString());
+        response.setReachable(reachable);
+        response.setLastSeen(lastActivityTime);
+
+        if (lastActivityTime != null) {
+            SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss", Locale.US);
+            response.setLastSeenFormatted(sdf.format(new Date(lastActivityTime)));
+            long minutesAgo = TimeUnit.MILLISECONDS.toMinutes(now - lastActivityTime);
+            response.setMessage(reachable
+                    ? "Device is online. Last activity " + minutesAgo + " minute(s) ago."
+                    : "Device is offline. Last activity " + minutesAgo + " minute(s) ago.");
+        } else {
+            response.setLastSeenFormatted("Never");
+            response.setMessage("Device has never been active.");
+        }
+
+        return response;
+    }
+
+    Long getLastActivityTime(TenantId tenantId, DeviceId deviceId) {
+        try {
+            List<TsKvEntry> lastActivity = timeseriesService.findLatest(tenantId, deviceId, List.of(LAST_ACTIVITY_TIME_KEY)).get();
+            if (!lastActivity.isEmpty() && lastActivity.get(0).getValue() != null) {
+                return lastActivity.get(0).getLongValue().orElse(null);
+            }
+
+            List<TsKvEntry> latestTelemetry = timeseriesService.findAllLatest(tenantId, deviceId).get();
+            if (!latestTelemetry.isEmpty()) {
+                return latestTelemetry.stream()
+                        .map(TsKvEntry::getTs)
+                        .max(Comparator.naturalOrder())
+                        .orElse(null);
+            }
+            return null;
+        } catch (Exception e) {
+            log.debug("Failed to resolve last activity time for device {}", deviceId, e);
+            return null;
+        }
+    }
+}

application/src/main/java/org/thingsboard/server/service/device/DevicePingService.java

@@ -0,0 +1,26 @@
+/**
+ * Copyright © 2016-2025 The Thingsboard Authors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.thingsboard.server.service.device;
+
+import org.thingsboard.server.common.data.device.DevicePingResponse;
+import org.thingsboard.server.common.data.id.DeviceId;
+import org.thingsboard.server.common.data.id.TenantId;
+
+public interface DevicePingService {
+
+    DevicePingResponse pingDevice(TenantId tenantId, DeviceId deviceId);
+
+}

application/src/test/java/org/thingsboard/server/service/device/DefaultDevicePingServiceTest.java

@@ -0,0 +1,105 @@
+/**
+ * Copyright © 2016-2025 The Thingsboard Authors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.thingsboard.server.service.device;
+
+import com.google.common.util.concurrent.Futures;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.junit.jupiter.MockitoExtension;
+import org.springframework.test.util.ReflectionTestUtils;
+import org.thingsboard.server.common.data.id.DeviceId;
+import org.thingsboard.server.common.data.id.TenantId;
+import org.thingsboard.server.common.data.kv.BasicTsKvEntry;
+import org.thingsboard.server.common.data.kv.LongDataEntry;
+import org.thingsboard.server.common.data.kv.TsKvEntry;
+import org.thingsboard.server.dao.timeseries.TimeseriesService;
+
+import java.util.List;
+import java.util.UUID;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.when;
+
+@ExtendWith(MockitoExtension.class)
+class DefaultDevicePingServiceTest {
+
+    @Mock
+    TimeseriesService timeseriesService;
+
+    @InjectMocks
+    DefaultDevicePingService service;
+
+    private static final TenantId TENANT_ID = TenantId.fromUUID(UUID.fromString("ce31b8c0-da96-11f0-888b-7f458e3ae7a2"));
+    private static final DeviceId DEVICE_ID = new DeviceId(UUID.fromString("784f394c-42b6-435a-983c-b7beff2784f9"));
+
+    @Test
+    void pingDevice_shouldBeReachable_whenLastActivityWithinWindow() throws Exception {
+        ReflectionTestUtils.setField(service, "reachabilityWindowMinutes", 5L);
+
+        long now = System.currentTimeMillis();
+        TsKvEntry lastActivity = new BasicTsKvEntry(now - 60_000, new LongDataEntry("lastActivityTime", now - 60_000));
+
+        when(timeseriesService.findLatest(eq(TENANT_ID), eq(DEVICE_ID), any(List.class)))
+                .thenReturn(Futures.immediateFuture(List.of(lastActivity)));
+
+        var resp = service.pingDevice(TENANT_ID, DEVICE_ID);
+
+        assertThat(resp.getDeviceId()).isEqualTo(DEVICE_ID.getId().toString());
+        assertThat(resp.isReachable()).isTrue();
+        assertThat(resp.getLastSeen()).isNotNull();
+    }
+
+    @Test
+    void pingDevice_shouldNotBeReachable_whenLastActivityOutsideWindow() throws Exception {
+        ReflectionTestUtils.setField(service, "reachabilityWindowMinutes", 5L);
+
+        long now = System.currentTimeMillis();
+        TsKvEntry lastActivity = new BasicTsKvEntry(now - 10 * 60_000, new LongDataEntry("lastActivityTime", now - 10 * 60_000));
+
+        when(timeseriesService.findLatest(eq(TENANT_ID), eq(DEVICE_ID), any(List.class)))
+                .thenReturn(Futures.immediateFuture(List.of(lastActivity)));
+
+        var resp = service.pingDevice(TENANT_ID, DEVICE_ID);
+
+        assertThat(resp.isReachable()).isFalse();
+        assertThat(resp.getLastSeen()).isNotNull();
+    }
+
+    @Test
+    void pingDevice_shouldFallbackToLatestTs_whenNoLastActivityKey() throws Exception {
+        ReflectionTestUtils.setField(service, "reachabilityWindowMinutes", 5L);
+
+        when(timeseriesService.findLatest(eq(TENANT_ID), eq(DEVICE_ID), any(List.class)))
+                .thenReturn(Futures.immediateFuture(List.of()));
+
+        long ts1 = System.currentTimeMillis() - 3 * 60_000;
+        long ts2 = System.currentTimeMillis() - 2 * 60_000;
+        TsKvEntry e1 = new BasicTsKvEntry(ts1, new LongDataEntry("temperature", 25L));
+        TsKvEntry e2 = new BasicTsKvEntry(ts2, new LongDataEntry("humidity", 40L));
+
+        when(timeseriesService.findAllLatest(eq(TENANT_ID), eq(DEVICE_ID)))
+                .thenReturn(Futures.immediateFuture(List.of(e1, e2)));
+
+        var resp = service.pingDevice(TENANT_ID, DEVICE_ID);
+
+        assertThat(resp.getLastSeen()).isEqualTo(ts2);
+        assertThat(resp.isReachable()).isTrue();
+    }
+}