Skip to content

Instantly share code, notes, and snippets.

Migration guide to new Background Jobs backend

As of MISP version 2.4.151 we introduced a simpler way to handle background jobs without relying in CakeResque as this library is no longer mantained.

For the time being both background jobs backends will be supported, but we plan to phase out the CakeResque one in a near future.

The new backend requires Supervisor and some extra PHP packages.

This guide is intended for Ubuntu/Debian systems

Install requirements

@korrosivesec
korrosivesec / spark_tips_and_tricks.md
Created September 3, 2021 17:41 — forked from dusenberrymw/spark_tips_and_tricks.md
Tips and tricks for Apache Spark.

Spark Tips & Tricks

Misc. Tips & Tricks

  • If values are integers in [0, 255], Parquet will automatically compress to use 1 byte unsigned integers, thus decreasing the size of saved DataFrame by a factor of 8.
  • Partition DataFrames to have evenly-distributed, ~128MB partition sizes (empirical finding). Always err on the higher side w.r.t. number of partitions.
  • Pay particular attention to the number of partitions when using flatMap, especially if the following operation will result in high memory usage. The flatMap op usually results in a DataFrame with a [much] larger number of rows, yet the number of partitions will remain the same. Thus, if a subsequent op causes a large expansion of memory usage (i.e. converting a DataFrame of indices to a DataFrame of large Vectors), the memory usage per partition may become too high. In this case, it is beneficial to repartition the output of flatMap to a number of partitions that will safely allow for appropriate partition memory sizes, based upon the
@korrosivesec
korrosivesec / send-message.py
Created January 17, 2018 18:09 — forked from ser1zw/send-message.py
Send E-mail from eml file in Python
#!/usr/bin/env python
# -*- mode: python; coding: utf-8 -*-
import sys
import os.path
import smtplib
if len(sys.argv) <= 2:
print('Usage:')
print(' $ python ' + sys.argv[0] + ' mailfrom rcptto <emlfile>')
print('')