Skip to content

Commit ad8f7bc

Browse files
authored
Configure joblogs storage (#38, PR #34)
* adjust k8s yaml to provide a path for logs file generated by the joblogs feature; create an fsGroup to allow writing to the provided directory, adjust PVC manifest; alter cofig.md file by deleting a clause about /tmp dir since we have an appropriate security context now * changed the path in the logs_dir; removed redundant PV manifest since PVC is enough to mount a PV; adjusted config.md to contain info about securityContext
1 parent 7a81737 commit ad8f7bc

File tree

2 files changed

+4
-14
lines changed

2 files changed

+4
-14
lines changed

CONFIG.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ For Kubernetes, it is important to set resource limits.
4646
TODO: explain how to set limits, with default, project and spider specificity.
4747

4848
### [joblogs] section
49-
* `logs_dir` - a directory to store log files collected on k8s cluster (implemented only for Kubernetes). When configuring, keep in mind that in the Dockerfile the `USER` is set to `nobody` so not all directories are writable, but if you make a child directory under `/tmp` you won't encounter permission problems.
49+
* `logs_dir` - a directory to store log files collected on k8s cluster (implemented only for Kubernetes). If you are using a Persistent Volume, keep in mind, that the provided path should be mounted in the deployment manifest. Read and write permissions should be granted to allow actions with log files in the provided directory, thus a [securityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) was added to the deployment manifest.
5050

5151

5252

kubernetes.yaml

+3-13
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ spec:
1313
labels:
1414
app.kubernetes.io/name: scrapyd-k8s
1515
spec:
16+
securityContext:
17+
fsGroup: 1000
1618
serviceAccountName: scrapyd-k8s
1719
containers:
1820
- image: ghcr.io/q-m/scrapyd-k8s:latest
@@ -106,7 +108,7 @@ data:
106108
limits_memory = 0.5G
107109
108110
[joblogs]
109-
logs_dir = /tmp/scrapyd_k8s_logs
111+
logs_dir = /data/joblogs
110112
---
111113
apiVersion: v1
112114
kind: Secret
@@ -118,18 +120,6 @@ stringData:
118120
FOO_API_KEY: "1234567890abcdef"
119121
#---
120122
#apiVersion: v1
121-
#kind: PersistentVolume
122-
#metadata:
123-
# name: pv-volume
124-
#spec:
125-
# capacity:
126-
# storage: 5Gi
127-
# accessModes:
128-
# - ReadWriteOnce
129-
# hostPath:
130-
# path: "/mnt/data"
131-
#---
132-
#apiVersion: v1
133123
#kind: PersistentVolumeClaim
134124
#metadata:
135125
# name: pv-claim

0 commit comments

Comments
 (0)