Deadline 的 Web Service 自訂 Script 跟 REST心得

最近練習用 react 做了一個讓同事可以直接用網頁或手機看算圖的網站。

從超級難用的 deadline documentation 總結出一些心得。

Web Service

雖然 deadline 有提供 REST api,但功能有限,譬如取得 jobs 篩選的方式就只能靠 status,沒辦法找特定 user 的,而且取得的 job 資料量非常龐大,包含外掛參數什麼都有,讓來往的封包沒意義的龐大。
所幸 Web Service 有 Web Service Scripts 這個功能,讓使用者可以用 Python 自訂資訊,足以彌補這方面的缺陷。

Web Service Scripts 的資料夾就在 DeadlineRepository\scripts\WebService 底下,裡面已經有一些內建的 script 是給 deadline 本身自有的 APP 用的。
只要將 demo.py 放在裡面,就可以用 http://host:port/demo 執行該請求,詳細的方法都在上面連結文件中,這邊就不多述。

Script 範例

雖然資料夾下就有一些現成 script,但那些文件因為架構很嚴謹難以去解讀,所以使用我做上面那個網站所寫的其中一個 script。
這個 script 檔名是 jobs.py,只要使用者連上 http://host:port/jobs,就可以取得目前正在算圖或排隊中的 jobs。
那如果有指定使用者,譬如 http://host:port/jobs?username=elisha,就會取得該使用者送出的 jobs。

from Deadline.Scripting import *
import json


def __main__(dlArgs, qsArgs):
    filter_name = qsArgs.get('username', None) # 用qsArgs取得參數

    jobs = WebServiceUtils.GetJobs() # 取得全部的jobs

    # 開始過濾
    result = []
    for job in jobs:
        # 如果沒有指定使用者名稱,過濾非活動中的jobs
        if filter_name is None and job['Status'] not in ['Active', 'Pending']:
            continue
        # 如果有指定使用者,過濾非使用者的jobs
        if filter_name is not None and job['UserName'] != filter_name:
            continue

        # 將只有網站需要的屬性map到回應的物件
        obj = {
            'name': job['Name'],
            'batch': job['BatchName'],
            'plugin': job['PluginName'],
            'user': job['UserName'],
            'status': job['Status'],
            'priority': job['Priority'],
            'id': job['JobId'],
            'submitDate': job['SubmitDateTimeString'],
            'startDate': job['StartedDateTimeString'],
            'completedDate': job['CompletedDateTimeString'],
            'firstFrame': job['FirstFrame'],
            'lastFrame': job['LastFrame'],
            'taskCount': job['TaskCount'],
            'pool': job['Pool'],
            'completedTasks': job['JobCompletedTasks'],
            'queuedTasks': job['JobQueuedTasks'],
            'suspendedTasks': job['JobSuspendedTasks'],
            'renderingTasks': job['JobRenderingTasks'],
            'failedTasks': job['JobFailedTasks'],
            'pendingTasks': job['JobPendingTasks'],
            'errors': job['ErrorReports'],
            'singleProgress': job['SingleTaskProgress'],
        }
        result.append(obj)

    return json.dumps(result) # 也可以是tuple,typle第二個數值是status code,預設為200

WebServiceUtils 算是這類 script 的核心,不同於其他難搞的 deadline 模組,WebServiceUtils.GetJobs() 會取得帶有全部資訊的 job 字典陣列,只是這些 job 不是 Python 原生的字典物件,無法直接使用 json.dumps() 給序列化,所以屬性還需個別取出。

CORS 問題

如果在網站上另外對 deadline 伺服器發送請求會產生 CORS 的問題,而 webserive 應用程式本身已經封裝起來,目前是沒看到有設定可以開啟 CORS,所以架這網站時有另外使用 CORS Anywhere 的方法來解決。

job 與 task 字典內容

WebServiceUtils 取得的 job 跟 task 內容一般情況比較難看到,順便附上。

job 物件

"job": [
    "Name",
    "BatchName",
    "UserName",
    "RegionName",
    "Comment",
    "Department",
    "FramesList",
    "ChunkSize",
    "TaskCount",
    "Group",
    "Pool",
    "SecondaryPool",
    "Priority",
    "ConcurrentTasks",
    "LimitTasksToNumberOfCpus",
    "ReloadRenderer",
    "SynchronizeAllAuxiliaryFiles",
    "InterruptibleFlag",
    "InterruptiblePercentage",
    "SequentialJobFlag",
    "SuppressEvents",
    "EnableAutoTimeout",
    "EnableTimeoutsForScriptTasks",
    "EnableFrameTimeouts",
    "OnTaskTimeout",
    "MinRenderTime",
    "MinRenderTimeSeconds",
    "TaskTimeout",
    "TaskTimeoutSeconds",
    "StartJobTimeout",
    "StartJobTimeoutSeconds",
    "InitializePluginTimeout",
    "InitializePluginTimeoutSeconds",
    "JobDependencies",
    "IsFrameDependent",
    "FrameDependencyOffsetStart",
    "FrameDependencyOffsetEnd",
    "ResumeOnCompleteDependencies",
    "ResumeOnDeletedDependencies",
    "ResumeOnFailedDependencies",
    "JobDependencyPercentage",
    "IgnoreBadJobDetection",
    "OverrideAutoJobCleanup",
    "OverrideJobCleanup",
    "JobCleanupDays",
    "OverrideJobCleanupType",
    "OverrideJobFailureDetection",
    "FailureDetectionJobErrors",
    "OverrideTaskFailureDetection",
    "FailureDetectionTaskErrors",
    "OverrideNotificationMethod",
    "EmailNotification",
    "PopupNotification",
    "NotificationEmails",
    "NotificationTargets",
    "NotificationNote",
    "SendJobErrorWarning",
    "OnJobComplete",
    "Protected",
    "LimitGroups",
    "ListedSlaves",
    "WhitelistFlag",
    "MachineLimit",
    "MachineLimitProgress",
    "PreJobScript",
    "PostJobScript",
    "PreTaskScript",
    "PostTaskScript",
    "ScheduledType",
    "ScheduledDays",
    "ScheduledStartDateTime",
    "ScheduledStopDateTime",
    "MondayStartTime",
    "MondayStopTime",
    "TuesdayStartTime",
    "TuesdayStopTime",
    "WednesdayStartTime",
    "WednesdayStopTime",
    "ThursdayStartTime",
    "ThursdayStopTime",
    "FridayStartTime",
    "FridayStopTime",
    "SaturdayStartTime",
    "SaturdayStopTime",
    "SundayStartTime",
    "SundayStopTime",
    "ExtraInfo0",
    "ExtraInfo1",
    "ExtraInfo2",
    "ExtraInfo3",
    "ExtraInfo4",
    "ExtraInfo5",
    "ExtraInfo6",
    "ExtraInfo7",
    "ExtraInfo8",
    "ExtraInfo9",
    "ExtraInfoDictionary",
    "OverrideTaskExtraInfoNames",
    "TaskExtraInfoName0",
    "TaskExtraInfoName1",
    "TaskExtraInfoName2",
    "TaskExtraInfoName3",
    "TaskExtraInfoName4",
    "TaskExtraInfoName5",
    "TaskExtraInfoName6",
    "TaskExtraInfoName7",
    "TaskExtraInfoName8",
    "TaskExtraInfoName9",
    "CustomPluginDirectory",
    "CustomEventPluginDirectory",
    "RequiredAssets",
    "AWSPortalAssets",
    "ScriptDependencies",
    "CompletedFrames",
    "TotalRenderTimeTicks",
    "TotalRenderTime",
    "IsSubmitted",
    "IsBeingPurged",
    "SubmitMachineName",
    "SubmitUserName",
    "SubmitDateTimeString",
    "SubmitDateTime",
    "StartedDateTimeString",
    "StartedDateTime",
    "CompletedDateTimeString",
    "CompletedDateTime",
    "PluginName",
    "OutputDirectories",
    "OutputFileNames",
    "OutputTileFileNames",
    "MaintenanceJob",
    "MaintenanceJobStartFrame",
    "MaintenanceJobEndFrame",
    "TileJob",
    "TileJobFrame",
    "TileJobTileCount",
    "TileJobTilesInX",
    "TileJobTilesInY",
    "Status",
    "AuxiliarySubmissionFileNames",
    "BadSlaves",
    "InitialCompletedTaskIds",
    "InitialUncompletedTaskIds",
    "PluginInfoDictionary",
    "EventOptIns",
    "EnvironmentDictionary",
    "UseJobEnvironmentOnly",
    "ExtraInfoKeyValues",
    "Frames",
    "FirstFrame",
    "LastFrame",
    "CompletedChunks",
    "QueuedChunks",
    "SuspendedChunks",
    "RenderingChunks",
    "FailedChunks",
    "PendingChunks",
    "SingleTaskProgress",
    "LogReportFileNames",
    "LogReportLastWriteTime",
    "ErrorReports",
    "PluginDataFileName",
    "PluginDataFileSize",
    "JobName",
    "JobBatchName",
    "JobComment",
    "JobDepartment",
    "JobPool",
    "RemTimeThreshold",
    "JobSecondaryPool",
    "JobGroup",
    "JobPriority",
    "JobUserName",
    "JobMinRenderTimeSeconds",
    "JobTaskTimeoutSeconds",
    "JobStartJobTimeoutSeconds",
    "JobInitializePluginTimeoutSeconds",
    "JobOnTaskTimeout",
    "JobEnableAutoTimeout",
    "JobEnableTimeoutsForScriptTasks",
    "JobEnableFrameTimeouts",
    "JobConcurrentTasks",
    "JobLimitTasksToNumberOfCpus",
    "JobOnJobComplete",
    "JobProtected",
    "JobScheduledType",
    "JobScheduledDays",
    "JobScheduledStartDateTime",
    "JobScheduledStopDateTime",
    "JobMondayStartTime",
    "JobMondayStopTime",
    "JobTuesdayStartTime",
    "JobTuesdayStopTime",
    "JobWednesdayStartTime",
    "JobWednesdayStopTime",
    "JobThursdayStartTime",
    "JobThursdayStopTime",
    "JobFridayStartTime",
    "JobFridayStopTime",
    "JobSaturdayStartTime",
    "JobSaturdayStopTime",
    "JobSundayStartTime",
    "JobSundayStopTime",
    "JobInterruptible",
    "JobInterruptiblePercentage",
    "JobForceReloadPlugin",
    "JobSequentialJob",
    "JobSynchronizeAllAuxiliaryFiles",
    "JobSuppressEvents",
    "JobOverrideNotificationMethod",
    "JobEmailNotification",
    "JobPopupNotification",
    "JobNotificationNote",
    "JobOverrideAutoJobCleanup",
    "JobOverrideJobCleanup",
    "JobOverrideJobCleanupDays",
    "JobOverrideJobCleanupType",
    "JobSendJobErrorWarning",
    "JobOverrideJobFailureDetection",
    "JobFailureDetectionJobErrors",
    "JobOverrideTaskFailureDetection",
    "JobFailureDetectionTaskErrors",
    "JobIgnoreBadSlaveDetection",
    "JobPreJobScript",
    "JobPostJobScript",
    "JobPreTaskScript",
    "JobPostTaskScript",
    "JobIsFrameDependent",
    "JobFrameDependencyOffsetStart",
    "JobFrameDependencyOffsetEnd",
    "JobResumeOnCompleteDependencies",
    "JobResumeOnDeletedDependencies",
    "JobResumeOnFailedDependencies",
    "JobDependencyPercentageValue",
    "JobUseJobEnvironmentOnly",
    "JobCustomPluginDirectory",
    "JobCustomEventPluginDirectory",
    "JobExtraInfo0",
    "JobExtraInfo1",
    "JobExtraInfo2",
    "JobExtraInfo3",
    "JobExtraInfo4",
    "JobExtraInfo5",
    "JobExtraInfo6",
    "JobExtraInfo7",
    "JobExtraInfo8",
    "JobExtraInfo9",
    "JobOverrideTaskExtraInfoNames",
    "JobTaskExtraInfoName0",
    "JobTaskExtraInfoName1",
    "JobTaskExtraInfoName2",
    "JobTaskExtraInfoName3",
    "JobTaskExtraInfoName4",
    "JobTaskExtraInfoName5",
    "JobTaskExtraInfoName6",
    "JobTaskExtraInfoName7",
    "JobTaskExtraInfoName8",
    "JobTaskExtraInfoName9",
    "JobCompletedTasks",
    "JobQueuedTasks",
    "JobSuspendedTasks",
    "JobRenderingTasks",
    "JobFailedTasks",
    "JobPendingTasks",
    "JobSubmitMachine",
    "JobSubmitDateTime",
    "JobStartedDateTime",
    "JobCompletedDateTime",
    "JobId",
    "JobStatus",
    "JobMaintenanceJob",
    "JobMaintenanceJobStartFrame",
    "JobMaintenanceJobEndFrame",
    "JobTileJob",
    "JobTileJobFrame",
    "JobTileJobTileCount",
    "JobTileJobTilesInX",
    "JobTileJobTilesInY",
    "JobLimitGroups",
    "JobListedSlaves",
    "JobWhitelistFlag",
    "JobMachineLimit",
    "JobMachineLimitProgress",
    "JobNotificationTargets",
    "JobNotificationEmails",
    "JobAuxiliarySubmissionFileNames",
    "JobOutputDirectories",
    "JobOutputFileNames",
    "JobOutputTileFileNames",
    "JobFramesPerTask",
    "JobFrames",
    "JobFramesList",
    "JobPlugin",
    "JobTaskCount",
    "JobDependencyIDs",
    "JobRequiredAssets",
    "JobScriptDependencies",
    "LastWriteTime",
    "ID",
    "CouchRevision",
    "CouchbaseCAS"
]

task 物件

 "task": [
    "JobID",
    "TaskID",
    "FrameString",
    "SlaveName",
    "Status",
    "Progress",
    "RenderStatus",
    "ErrorCount",
    "TaskStartTime",
    "RenderStartTime",
    "CompletedDateTime",
    "Name",
    "IsStarting",
    "WaitingToStart",
    "RenderTime",
    "TaskTime",
    "RenderTimeMultiplier",
    "NormalizedRenderTime",
    "ImageFileSize",
    "PeakRamUsage",
    "PeakRamPercentage",
    "PeakCpuUsage",
    "CpuUtilisation",
    "AverageRam",
    "AverageRamPercentage",
    "AverageSwap",
    "PeakSwap",
    "UsedCpuClocks",
    "TotalCpuClocks",
    "TaskId",
    "TaskFrameString",
    "TaskFrameList",
    "TaskStatus",
    "TaskRenderStatus",
    "TaskAverageRam",
    "TaskAverageRamPercentage",
    "TaskAverageSwap",
    "TaskCompletedDateTime",
    "TaskCpuUtilisation",
    "TaskErrorCount",
    "TaskOutputFileSize",
    "TaskIsStarted",
    "TaskJobId",
    "TaskName",
    "TaskNormalizedRenderTime",
    "TaskPeakCpuUsage",
    "TaskPeakRamPercentage",
    "TaskPeakRamUsage",
    "TaskPeakSwap",
    "TaskProgress",
    "TaskRenderStartTime",
    "TaskRenderTime",
    "TaskRenderTimeMultiplier",
    "TaskSlaveName",
    "TaskTotalCpuClocks",
    "TaskUsedCpuClocks",
    "TaskWaitingToStart",
    "TaskProperties",
    "ID",
    "CouchRevision",
    "CouchbaseCAS"
]