Awesome List Updates on Nov 20 - Nov 26, 2023

26 awesome lists updated this week.

🏠 Home · 🔍 Search · 🔥 Feed · 📮 Subscribe · ❤️ Sponsor

1. Awesome Ada

Frameworks / Apache License

• adamant (⭐36) - A component-based, model-driven framework for constructing reliable and reusable real-time software.

2. Awesome Agi Cocosci

Communications / Non-Verbal Communication

• Twelve-month-olds communicate helpfully and appropriately for knowledgeable and ignorant partners - Cognition, 2008. [All Versions]. The original paper on child pointing.

• 12- and 18-Month-Olds Point to Provide Information for Others - Journal of Cognition and Development, 2009. [All Versions].

3. Awesome Db Tools

Data / Generation/Masking/Subsetting

• Synthesized TDK - DevOps' best friend for database masking and generation.

4. Awesome Cpp

GUI

• FOX Toolkit - an open-source, cross-platform widget toolkit. [LGPL]

Miscellaneous

• Jinja2С++ (⭐550) - almost full-conformance template engine implementation. website

• scnlib (⭐1.2k) - scanf for modern C++. [Apache-2.0] website

5. Awesome Quant

Python / Trading & Backtesting

• Investing algorithm framework (⭐471) - Framework for developing, backtesting, and deploying automated trading algorithms.

6. Awesome Cl

System interface / Third-party APIs

• machine-state (⭐26) - Retrieve machine state information about CPU time, memory usage, thread processing time, etc.

7. Awesome Mac

Translation Tools / Audio Record and Process

• Easydict (⭐9.8k) - Easy to look up words or translate text

8. Ai Collective Tools

Audio Editing

• CrystalSound - Experience noise-free audio with AI.#freemium

• Gladia - Speech-to-Text API. #freemium

Design Generator

• Canva - Use it to create social media posts, presentations, posters, videos, logos and more. #freemium

• Designs AI - Make a logo, video, social media banner, business card, flyer, mockup and more with AI. #freemium

9. Awesome Data Engineering

Databases

• Relational
  • RQLite (⭐17k) - Replicated SQLite using the Raft consensus protocol.
  • MySQL - The world's most popular open source database.
    • TiDB (⭐39k) - TiDB is a distributed NewSQL database compatible with MySQL protocol.
    • Percona XtraBackup - Percona XtraBackup is a free, open source, complete online backup solution for all versions of Percona Server, MySQL® and MariaDB®.
    • mysql_utils (⭐885) - Pinterest MySQL Management Tools.
  • MariaDB - An enhanced, drop-in replacement for MySQL.
  • PostgreSQL - The world's most advanced open source database.
  • Amazon RDS - Amazon RDS makes it easy to set up, operate, and scale a relational database in the cloud.
  • Crate.IO - Scalable SQL database with the NOSQL goodies.

• Key-Value
  • Redis - An open source, BSD licensed, advanced key-value cache and store.
  • Riak - A distributed database designed to deliver maximum data availability by distributing data across multiple servers.
  • AWS DynamoDB - A fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale.
  • HyperDex (⭐1.4k) - HyperDex is a scalable, searchable key-value store. Deprecated.
  • SSDB - A high performance NoSQL database supporting many data structures, an alternative to Redis.
  • Kyoto Tycoon (⭐280) - Kyoto Tycoon is a lightweight network server on top of the Kyoto Cabinet key-value database, built for high-performance and concurrency.
  • IonDB (⭐593) - A key-value store for microcontroller and IoT applications.

• Column
  • Cassandra - The right choice when you need scalability and high availability without compromising performance.
    • Cassandra Calculator - This simple form allows you to try out different values for your Apache Cassandra cluster and see what the impact is for your application.
    • CCM (⭐1.2k) - A script to easily create and destroy an Apache Cassandra cluster on localhost.
    • ScyllaDB (⭐15k) - NoSQL data store using the seastar framework, compatible with Apache Cassandra.
  • HBase - The Hadoop database, a distributed, scalable, big data store.
  • AWS Redshift - A fast, fully managed, petabyte-scale data warehouse that makes it simple and cost-effective to analyze all your data using your existing business intelligence tools.
  • FiloDB (⭐1.4k) - Distributed. Columnar. Versioned. Streaming. SQL.
  • Vertica - Distributed, MPP columnar database with extensive analytics SQL.
  • ClickHouse - Distributed columnar DBMS for OLAP. SQL.

• Document
  • MongoDB - An open-source, document database designed for ease of development and scaling.
    • Percona Server for MongoDB - Percona Server for MongoDB® is a free, enhanced, fully compatible, open source, drop-in replacement for the MongoDB® Community Edition that includes enterprise-grade features and functionality.
    • MemDB (⭐593) - Distributed Transactional In-Memory Database (based on MongoDB).
  • Elasticsearch - Search & Analyze Data in Real Time.
  • Couchbase - The highest performing NoSQL distributed database.
  • RethinkDB - The open-source database for the realtime web.
  • RavenDB - Fully Transactional NoSQL Document Database.

• Graph
  • Neo4j - The world's leading graph database.
  • OrientDB - 2nd Generation Distributed Graph Database with the flexibility of Documents in one product with an Open Source commercial friendly license.
  • ArangoDB - A distributed free and open-source database with a flexible data model for documents, graphs, and key-values.
  • Titan - A scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster.
  • FlockDB (⭐3.3k) - A distributed, fault-tolerant graph database by Twitter. Deprecated.

• Distributed
  • DAtomic - The fully transactional, cloud-ready, distributed database.
  • Apache Geode - An open source, distributed, in-memory database for scale-out applications.
  • Gaffer (⭐1.8k) - A large-scale graph database.

• Timeseries
  • InfluxDB (⭐30k) - Scalable datastore for metrics, events, and real-time analytics.
  • OpenTSDB (⭐5.1k) - A scalable, distributed Time Series Database.
  • QuestDB - A relational column-oriented database designed for real-time analytics on time series and event data.
  • kairosdb (⭐1.7k) - Fast scalable time series database.
  • Heroic (⭐846) - A scalable time series database based on Cassandra and Elasticsearch, by Spotify.
  • Druid (⭐14k) - Column oriented distributed data store ideal for powering interactive applications.
  • Riak-TS - Riak TS is the only enterprise-grade NoSQL time series database optimized specifically for IoT and Time Series data.
  • Akumuli (⭐836) - Akumuli is a numeric time-series database. It can be used to capture, store and process time-series data in real-time. The word "akumuli" can be translated from esperanto as "accumulate".
  • Rhombus - A time-series object store for Cassandra that handles all the complexity of building wide row indexes.
  • Dalmatiner DB (⭐691) - Fast distributed metrics database.
  • Blueflood (⭐597) - A distributed system designed to ingest and process time series data.
  • Timely (⭐385) - Timely is a time series database application that provides secure access to time series data based on Accumulo and Grafana.

Data Ingestion

• AWS Kinesis - A fully managed, cloud-based service for real-time data processing over large, distributed data streams.

• RabbitMQ - Robust messaging for applications.

• FluentD - An open source data collector for unified logging layer.

• Embulk - An open source bulk data loader that helps data transfer between various databases, storages, file formats, and cloud services.

• Apache Sqoop - A tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases.

• Heka (⭐3.4k) - Data Acquisition and Processing Made Easy. Deprecated.

• Nakadi - Nakadi is an open source event messaging platform that provides a REST API on top of Kafka-like queues.

• Pravega - Pravega provides a new storage abstraction - a stream - for continuous and unbounded data.

• Apache Pulsar - Apache Pulsar is an open-source distributed pub-sub messaging system.

• Airbyte - Open-source data integration for modern data teams.

File System

• HDFS - A distributed file system designed to run on commodity hardware.
  • Snakebite (⭐857) - A pure python HDFS client.

• AWS S3 - Object storage built to retrieve any amount of data from anywhere.
  • smart_open (⭐3.3k) - Utils for streaming large files (S3, HDFS, gzip, bz2).

• Alluxio - Alluxio is a memory-centric distributed storage system enabling reliable data sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.

• OrangeFS - Orange File System is a branch of the Parallel Virtual File System.

• GlusterFS - Gluster Filesystem.

• XtreemFS - Fault-tolerant distributed file system for all storage needs.

• SeaweedFS (⭐25k) - Seaweed-FS is a simple and highly scalable distributed file system. There are two objectives: to store billions of files! to serve the files fast! Instead of supporting full POSIX file system semantics, Seaweed-FS choose to implement only a key~file mapping. Similar to the word "NoSQL", you can call it as "NoFS".

• S3QL (⭐1.2k) - S3QL is a file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack.

• LizardFS - LizardFS Software Defined Storage is a distributed, parallel, scalable, fault-tolerant, Geo-Redundant and highly available file system.

Serialization format

• Apache Avro - Apache Avro™ is a data serialization system.

• Apache Parquet - Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.
  • Snappy (⭐6.4k) - A fast compressor/decompressor. Used with Parquet.
  • PigZ - A parallel implementation of gzip for modern multi-processor, multi-core machines.

• Apache ORC - The smallest, fastest columnar storage for Hadoop workloads.

• Apache Thrift - The Apache Thrift software framework, for scalable cross-language services development.

• ProtoBuf (⭐68k) - Protocol Buffers - Google's data interchange format.

• SequenceFile - SequenceFile is a flat file consisting of binary key/value pairs. It is extensively used in MapReduce as input/output formats.

• Kryo (⭐6.4k) - Kryo is a fast and efficient object graph serialization framework for Java.

Stream Processing

• Apache Beam - Apache Beam is a unified programming model that implements both batch and streaming data processing jobs that run on many execution engines.

• Spark Streaming - Spark Streaming makes it easy to build scalable fault-tolerant streaming applications.

• Apache Flink - Apache Flink is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.

• Apache Storm - Apache Storm is a free and open source distributed realtime computation system.

• Apache Samza - Apache Samza is a distributed stream processing framework.

• Apache NiFi - An easy to use, powerful, and reliable system to process and distribute data.

• Apache Hudi - An open source framework for managing storage for real time processing, one of the most interesting feature is the Upsert.

• VoltDB - VoltDb is an ACID-compliant RDBMS which uses a shared nothing architecture.

• PipelineDB (⭐2.6k) - The Streaming SQL Database.

• Spring Cloud Dataflow - Streaming and tasks execution between Spring Boot apps.

• Bonobo - Bonobo is a data-processing toolkit for python 3.5+.

• Robinhood's Faust (⭐1.8k) - Forever scalable event processing & in-memory durable K/V store as a library with asyncio & static typing.

• HStreamDB (⭐723) - The streaming database built for IoT data storage and real-time processing.

• Kuiper (⭐1.6k) - An edge lightweight IoT data analytics/streaming software implemented by Golang, and it can be run at all kinds of resource-constrained edge devices.

Batch Processing

• Hadoop MapReduce - Hadoop MapReduce is a software framework for easily writing applications which process vast amounts of data (multi-terabyte data-sets) - in-parallel on large clusters (thousands of nodes) - of commodity hardware in a reliable, fault-tolerant manner.

• Spark - A multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
  • Spark Packages - A community index of packages for Apache Spark.
  • Deep Spark (⭐197) - Connecting Apache Spark with different data stores. Deprecated.
  • Spark RDD API Examples - Examples by Zhen He.
  • Livy - The REST Spark Server.
  • Delight (⭐344) - A free & cross platform monitoring tool (Spark UI / Spark History Server alternative).

• AWS EMR - A web service that makes it easy to quickly and cost-effectively process vast amounts of data.

• Data Mechanics - A cloud-based platform deployed on Kubernetes making Apache Spark more developer-friendly and cost-effective.

• Tez - An application framework which allows for a complex directed-acyclic-graph of tasks for processing data.

• Bistro (⭐7) - A light-weight engine for general-purpose data processing including both batch and stream analytics. It is based on a novel unique data model, which represents data via functions and processes data via columns operations as opposed to having only set operations in conventional approaches like MapReduce or SQL.

• Batch ML
  • H2O - Fast scalable machine learning API for smarter applications.
  • Mahout - An environment for quickly creating scalable performant machine learning applications.
  • Spark MLlib - Spark's scalable machine learning library consisting of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as underlying optimization primitives.

• Batch Graph
  • GraphLab Create - A machine learning platform that enables data scientists and app developers to easily create intelligent apps at scale.
  • Giraph - An iterative graph processing system built for high scalability.
  • Spark GraphX - Apache Spark's API for graphs and graph-parallel computation.

• Batch SQL
  • Presto - A distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources.
  • Hive - Data warehouse software facilitates querying and managing large datasets residing in distributed storage.
    • Hivemall (⭐312) - Scalable machine learning library for Hive/Hadoop.
    • PyHive (⭐1.7k) - Python interface to Hive and Presto.
  • Drill - Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage.

Charts and Dashboards

• Highcharts - A charting library written in pure JavaScript, offering an easy way of adding interactive charts to your web site or web application.

• ZingChart - Fast JavaScript charts for any data set.

• C3.js - D3-based reusable chart library.

• SmoothieCharts - A JavaScript Charting Library for Streaming Data.

• PyXley (⭐2.3k) - Python helpers for building dashboards using Flask and React.

• Plotly (⭐23k) - Flask, JS, and CSS boilerplate for interactive, web-based visualization apps in Python.

• Apache Superset (⭐67k) - Apache Superset (incubating) - A modern, enterprise-ready business intelligence web application.

• Redash - Make Your Company Data Driven. Connect to any data source, easily visualize and share your data.

• Metabase (⭐43k) - Metabase is the easy, open source way for everyone in your company to ask questions and learn from data.

• PyQtGraph - PyQtGraph is a pure-python graphics and GUI library built on PyQt4 / PySide and numpy. It is intended for use in mathematics / scientific / engineering applications.

Workflow

• Luigi (⭐18k) - Luigi is a Python module that helps you build complex pipelines of batch jobs.

• Cascading - Java based application development platform.

• Oozie - Oozie is a workflow scheduler system to manage Apache Hadoop jobs.

• Dagster (⭐14k) - Dagster is an open-source Python library for building data applications.

• Kedro - Kedro is a framework that makes it easy to build robust and scalable data pipelines by providing uniform project templates, data abstraction, configuration and pipeline assembly.

• Dataform - An open-source framework and web based IDE to manage datasets and their dependencies. SQLX extends your existing SQL warehouse dialect to add features that support dependency management, testing, documentation and more.

• Census - A reverse-ETL tool that let you sync data from your cloud data warehouse to SaaS applications like Salesforce, Marketo, HubSpot, Zendesk, etc. No engineering favors required—just SQL.

• dbt - A command line tool that enables data analysts and engineers to transform data in their warehouses more effectively.

• RudderStack (⭐4.2k) - A warehouse-first Customer Data Platform that enables you to collect data from every application, website and SaaS platform, and then activate it in your warehouse and business tools.

Data Lake Management

• lakeFS (⭐4.8k) - lakeFS is an open source platform that delivers resilience and manageability to object-storage based data lakes.

ELK Elastic Logstash Kibana

• elasticsearch-jdbc (⭐2.8k) - JDBC importer for Elasticsearch.

• ZomboDB (⭐4.7k) - Postgres Extension that allows creating an index backed by Elasticsearch.

Docker

• Flocker (⭐3.4k) - Easily manage Docker containers & their data.

• Rancher - RancherOS is a 20mb Linux distro that runs the entire OS as Docker containers.

• Kontena - Application Containers for Masses.

• Weave (⭐6.6k) - Weaving Docker containers into applications.

• Zodiac (⭐199) - A lightweight tool for easy deployment and rollback of dockerized applications.

• cAdvisor (⭐18k) - Analyzes resource usage and performance characteristics of running containers.

• Micro S3 persistence (⭐14) - Docker microservice for saving/restoring volume data to S3.

• Rocker-compose (⭐410) - Docker composition tool with idempotency features for deploying apps composed of multiple containers. Deprecated.

Datasets / Realtime

• Twitter Realtime - The Streaming APIs give developers low latency access to Twitter's global stream of Tweet data.

• Eventsim (⭐524) - Event data simulator. Generates a stream of pseudo-random events from a set of users, designed to simulate web traffic.

• Reddit - Real-time data is available including comments, submissions and links posted to reddit.

Datasets / Data Dumps

• GitHub Archive - GitHub's public timeline since 2011, updated every hour.

• Common Crawl - Open source repository of web crawl data.

Monitoring / Prometheus

• Prometheus.io (⭐59k) - An open-source service monitoring system and time series database.

• HAProxy Exporter (⭐621) - Simple server that scrapes HAProxy stats and exports them via HTTP for Prometheus consumption.

Testing / Data Profiler

• Grai (⭐307) - A data catalog tool that integrates into your CI system exposing downstream impact testing of data changes. These tests prevent data changes which might break data pipelines or BI dashboards from making it to production.

Community / Forums

• /r/etl - Subreddit focused on ETL.

Community / Conferences

• Data Council - Data Council is the first technical conference that bridges the gap between data scientists, data engineers and data analysts.

Community / Podcasts

• Data Engineering Podcast - The show about modern data infrastructure.

10. Awesome K6

Articles

• Load testing with k6 and k8s - The Toucan DevOps team explains why they chose k6 as their load testing tool and how they deploy it on k8s.

• CloudPosse's Load Testing Stack (⭐56) - Load testing stack using k6, Grafana and InfluxDB.

• Real-time stress: AnyCable, k6, WebSockets, and Yabeda - Evil Martians adding "real-time stress" with k6 and WebSockets.

• Scaling Confidently with the Load and Fault Team - Robinhood on load testing a Kubernetes system with k6.

Extensions

• GitHub Topic: xk6 - Explore k6 extensions tagged with the xk6 label.

Related / Community

• How They Load Test (⭐124) - A collection of resources on how companies around the world perform load testing.

• Load Testing Toolkit (⭐203) - A collection of open-source tools for debugging, benchmarking, load and stress testing your code or services.

• awesome-http-benchmark (⭐3.6k) - A collection of HTTP benchmark tools, testing/debugging, & restAPI (RESTful).

11. Free Podcasts/Screencasts (English)

C++

• C++ STL: The ONLY Video You Need | Compulsory for DSA/CP - Utkarsh Gupta (screencast)

12. Awesome Zsh Plugins

Plugins / superconsole - Windows-only

• take (⭐3) - Replicates take from oh-my-zsh.

Themes / superconsole - Windows-only

• achab (⭐3) - Includes decorators for the current folder path, the current user and the current git branch. Decorator colors are easily customizable.

• htb (⭐2) - Inspired by the pwnbox on a HackTheBox machine. Includes decorators for user@host, IP address and the current directory.

• ruweird (⭐0) - Minimalist. Has decorators for git status and current directory. Shows an umbrella with raindrops and exit code of the last command if non-zero.

• velvet (⭐0) - Includes decorators for git status, username, current directory, exit status of last command, and the time.

13. Awesome Jmeter

CI / Tools & Plugins

• Lightning - Framework to integrate JMeter non-functional tests with CI/CD server.

Distributed Testing / Tutorials & Demo

• Dockerized
  • Dockerized JMeter - Distributed load testing workflow with Docker and JMeter.
  • JMeter Docker Images
  • Distributed JMeter testing using Docker
  • A Docker solution to JMeter + InfluxDB + Grafana performance testing
  • AutoMeter (⭐60) - An automation tool for scaling load tests using distributed slaves, based on JMeter master-slave architecture.
  • JMeter Docker Extension - Docker extension to run JMeter tests from Docker Desktop.

Tools / Utilities

• Hamster (⭐16) - Swiftly launch your JMeter test plans from Mac menubar.

14. Awesome Datascience

Other Awesome Lists / Book Deals (Affiliated)

• How Generative AI Is Changing Creative Work

• What is generative AI?

15. Awesome Selfhosted

Software / Communication - Email - Mail Transfer Agents

• OpenTrashmail (⭐701) - Complete trashmail solution that exposes an SMTP server and has a web interface to manage received emails. Works with multiple and wildcard domains and is fully file based (no database needed). Includes RSS feeds and JSON API. Apache-2.0 Python/PHP/Docker

Software / Groupware

• Digiwall - Create multimedia collaborative walls for in-person or remote work (documentation in French). (Source Code) AGPL-3.0 Nodejs

16. Awesome Qubes OS

System Information & Requirements

• System Requirements - User documentation / Choyhe attacker doesn’t have access to all the software running in the other domains.

• Secondary storage - hese steps assume you have already created a separate volume group and thin pool (not thin volume) for your HDD.

• Installing contributed packages - This page is for users who wish to install contributed packages.

How-to guides

• How to persist Tailscale tunnel in QubesOS (⭐2) - Tailscale is a mesh private network that lets you easily manage access to private resources or quickly SSH into devices on your network.

• How to use Qubes for journalists and Humans Rights Defenders (⭐20) - A suite of tools for improving and streamlining the use of Qubes OS for reporters, edtiors, lawyers and aid workers. Work in progress.
  • PAM distress login for xscreensaver - A simple setup for sending a distress email if forced to login to desktop beyond LUKS passwords.
  • Automated Arkenfox Qubes browser install (⭐20) - Arkenfox config install for journalists

• How to Implement TPM Boot Verification - Verify and validate boot process using PCR without Heads or AEM on systems using TPM 2.0

• How to open Urls in other qube - This page is about opening URLs and files from one qube in a different qube.

Templates

• archlinux-minimal template - This is a community guide, not an official guide.

• audio-qubes - An audio qube acts as a secure handler for potentially malicious audio devices, preventing them from coming into contact with dom0

• Building a TemplateVM for a new OS - If you don’t like using one of the existing templates because of specific administration, package management or other building needs, you can build a TemplateVM for your distribution of choice.

• NetBSD templates - Createa NetBSD template

• Linux HVMs - Fixing Linux distro HVMs

• Multimedia template - Configuring a “Multimedia” TemplateVM

• Pentesting: BlackArch - BlackArch Linux is an Arch Linux-based distribution for penetration testers and security researchers.

• Pentesting: PTF - "The PenTesters Framework (PTF) is a Python script designed for Debian/Ubuntu/ArchLinux based distributions to create a similar and familiar distribution for Penetration Testing.

• Windows Qubes - Like any other unmodified OSes, Windows can be installed in Qubes as an HVM domain.

• Qvm-Create-Windows-Qube (⭐363) - Qvm-Create-Windows-Qube is a tool for quickly and conveniently installing fresh new Windows qubes with Qubes Windows Tools (QWT) drivers automatically.

• Tails OS - Tails stands for The Amnesic Incognito Live System. It is a live operating system that aims to preserve your privacy and anonymit.

• Shadow qube (⭐7) - The below script will create a Qube, launch the Tor browser, wait for the browser to close, then remove the qube and its RAM pool.

• USB Qubes - A USB qube acts as a secure handler for potentially malicious USB devices, preventing them from coming into contact with dom0 (which could otherwise be fatal to the security of the whole system). I

VM-Hardening

• Dom0 secure updates - Updating dom0

• nft Firewall - This is an example for a TCP redirection, for UDP you would have to replace tcp by udp.

• Signal - It uses end-to-end encryption to secure all communications.

• Split SSH - Split SSH implements a concept similar to having a smart card with your private SSH keys, except that the role of the “smart card” is played by another Qubes AppVM.

• Qrexec: secure communication across domains - The qrexec framework is used by core Qubes components to implement communication between domains.

• YubiKey - Most use cases for the YubiKey can be achieved exactly as described by the manufacturer or other instructions found online.

Customization

• Bash completion - How to install bash completion for Qubes OS commands.

• QubesOS Autostart Menu (⭐0) - Speed up system boot process with a custom launch script

• Qubes-Scripts (⭐18) - Collection of custom scripts for Qubes OS.

• sys-VPN notification setup - Get VPN stats as a desktop notification

• Wayland agent (⭐21) - This is a GUI agent for Qubes OS that supports the [Wayland] display server protocol. Compared to X11, Wayland is vastly simpler and aims to ensure every frame is perfect.

Troubleshooting

• Mounting a qube’s private storage in another qube - Useful for data recovery.

• Traffic Analysis in Qubes OS - You can place a ProxyVM between your AppVMs and Your NetVM. This way we can create an ideal topology for traffic analysis.

• Analyze Qubes OS VM (⭐1) - Analyze Qubes OS VM startup performance.

• Url redirector (⭐36) - This is a browser extension, aimed to improve surfing security.

• Changing your Time Zone - Change the system’s time zone in terminal you can issue the timedatectl command with the option set-timezone.

Clearnet & Anonymous Networking / DNS

• dnscrypt-proxy - Run dnscrypt-proxy inside of sys-net to encrypt and secure dns-requests.

Clearnet & Anonymous Networking / Wireguard

• Mullvad VPN (Fedora38 + WG) - Privacy-first VPN provider's guide for Qubes OS. This guide bears an optimal method for setting up a WG ProxyVM (i.e sys-vpn); you may substitute out Mullvad's WG configuration files in place of your own.

• Wireguard setup - This guide assumes you are using a VPN service that has wireguard support.

Clearnet & Anonymous Networking / OpenVPN

• Mullvad VPN (Dedian 12 + OVPN) - Privacy-first VPN provider's guide for Qubes OS.

Clearnet & Anonymous Networking / Anonymity

• Really disposable ram based qubes - You can use your QubesOS 𝚜𝚝𝚊𝚝𝚎𝚕𝚎𝚜𝚜 just like TailsOS, with persistent storage for VMs. That is pretty simple! It takes 6Gb of extra 𝚁𝙰𝙼 (for store root filesystem files).

• How to bypass the GFW on Qubes OS when you’re in China - The purpose of this article is to provide several feasible ways to bypass the GFW for you to choose.

Exploitation Tools / Unikernel-like

• Hyperdbg (⭐3.3k) - HyperDbg Debugger is an open-source, community-driven, hypervisor-assisted, user-mode, and kernel-mode Windows debugger with a focus on using modern hardware technologies. It is a debugger designed for analyzing, fuzzing, and reversing.

• Capstone - Capstone is a lightweight multi-platform, multi-architecture disassembly framework.

17. Awesome Speaking

Videos

• KubeCon + CloudNativeCon 2020 Keynote: Hiding in the Dark - by Dan Kohn [07:51]

18. Awesome Polars

Blog posts / Miscellaneous

• Working with DateTime data in Polars - A blog post to helps you with the main operations that can be done with datetime data by Rielly Griffiths.

• Revolutionize Your Data Analysis: Polars Outperforms Pandas by Up to 5x in Numerical Filter Operations! - A blog post that compares Polars with Pandas by examining their performance in the real world by Daniel Builescu.

19. Awesome Swift

Misc

• MemberwiseInit (⭐150) - @MemberwiseInit is a Swift Macro that can more often provide your intended init, while following the same safe-by-default semantics of Swift’s memberwise initializers.

Alert / Barcode

• FullscreenPopup (⭐38) - Present any popup above NavigationBar in SwiftUI

20. Awesome Generative Deep Art

Prompt Engineering

• [2310.04438] A Brief History of Prompt: Leveraging Language Models: the paper presents an exploration of the evolution of prompt engineering. The author, Golam Md Muktadir, extensively used ChatGPT for content generation

Critical Views about Generative AI

• [2308.03762] GPT-4 Can't Reason: "despite the genuinely impressive improvement, there are good reasons to be highly skeptical of GPT-4's ability to reason"

• Risk and Harm: Unpacking Ideologies in the AI Discourse | Proceedings of the 5th International Conference on Conversational User Interfaces

• [2305.18654] Faith and Fate: Limits of Transformers on Compositionality

• [2210.02667] A Human Rights-Based Approach to Responsible AI

• On the Dangers of Stochastic Parrots | Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

Large Language Models (LLMs)

• Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)

• Poe: a platform that lets people ask questions, get instant answers, and have back-and-forth conversations with a wide variety of AI-powered bots

LLM Evaluation / Multi-agents

• [2311.12983] GAIA: a benchmark for General AI Assistants

• Sharing LangSmith Benchmarks

• [2311.09247] Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks

• vectara/hallucination-leaderboard (⭐2.6k): "leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents"

Image Segmentation / Deforum

• [Emu Video | Meta ](https://emu-video.metademolab.com/demo#/demo): state-of-the-art text-to-video generation

• Mustango: "Toward Controllable Text-to-Music Generation"

• Lyria by Google DeepMind: "transforming the future of music creation"

• Suno AI: "make any song you can imagine"

• Riffusion: this AI system generates singing voice for literally any text as input

• Stable Audio - Generative AI for music & sound fx

Roadmaps, Tracks, Rails / Deforum

• (1166) A Hackers' Guide to Language Models - YouTube

21. Awesome Vite

Framework-agnostic Plugins / Transformers

• vite-plugin-public-typescript (⭐13) - Inject Typescript into index.html.

Framework-agnostic Plugins / Helpers

• vite-font-extractor-plugin (⭐6) - Utility that minification bundle fonts by glyphs.

• vite-plugin-i18n-detector (⭐20) - Lazyload i18n locale resources.

22. Awesome Neovim

AI / Diagnostics

• jpmcb/nvim-llama (⭐274) - LLM (Llama 2 and llama.cpp) wrappers.

• David-Kunz/gen.nvim (⭐1.5k) - Generate text using LLMs (via Ollama) with customizable prompts.

Snippet / Diagnostics

• rafamadriz/friendly-snippets (⭐2.4k) - Set of preconfigured snippets for different languages.

• cvigilv/esqueleto.nvim (⭐104) - Simple templates to use when creating new files.

Fuzzy Finder / Diagnostics

• fdschmidt93/telescope-egrepify.nvim (⭐134) - Telescope plugin for better rg flags in live_grep.

Debugging / Diagnostics

• theHamsta/nvim-dap-virtual-text (⭐984) - Virtual text support for nvim-dap.

Code Runner / Diagnostics

• bfredl/nvim-ipy (⭐428) - Make interfacing with IPython/Jupyter easier.

Motion / Diagnostics

• ggandor/flit.nvim (⭐382) - Enhanced f/t motions for Leap.

• ggandor/leap-spooky.nvim (⭐277) - Spooky (Leap) actions at a distance.

Editing Support / Diagnostics

• nat-418/boole.nvim (⭐179) - Toggle booleans and common string values.

• Jxstxs/conceal.nvim (⭐75) - Use Tree-sitter to conceal common boilerplate code.

Formatting / Diagnostics

• nvimdev/guard.nvim (⭐494) - Minimalist async formatting and linting plugin.

23. Awesome Directus

Resources / Community

• Directus Extensions - A searchable index of Directus extensions, themes, OSes, and more.

24. Awesome Dotfiles

Tools / Ansible

• comtrya (⭐563) - Configuration management for localhost, written in Rust, for Linux, BSD, macOS, and Windows.

• Fisher (⭐8.5k) - A package manager for Fish.

• mackup (⭐15k) - Keep your application settings in sync (macOS/Linux).

Tools / macOS

• dockutil (⭐1.5k) - Command line tool for managing dock items.

• mas (⭐11k) - Mac App Store command line interface.

Archive/abandoned projects / macOS

• macOS Defaults (⭐1.3k)

25. Awesome Fuzzing

Papers / ACM Conference on Computer and Communications Security (ACM CCS)

• Fuzz on the Beach: Fuzzing Solana Smart Contracts, 2023

• NestFuzz: Enhancing Fuzzing with Comprehensive Understanding of Input Processing Logic, 2023

• Profile-Driven System Optimizations for Accelerated Greybox Fuzzing, 2023

• Hopper: Interpretative Fuzzing for Libraries, 2023

• Greybox Fuzzing of Distributed Systems, 2023

Tools / API

• IvySyn - IvySyn is a fully-automated framework for discovering memory error vulnerabilities in Deep Learning (DL) frameworks.

• MINER (⭐39) - MINER is a REST API fuzzer that utilizes three data-driven designs working together to guide the sequence generation, improve the request generation quality, and capture the unique errors caused by incorrect parameter usage.

• RestTestGen (⭐55) - RestTestGen is a robust tool and framework designed for automated black-box testing of RESTful web APIs.

• GraphFuzz (⭐10) - GraphFuzz is an experimental framework for building structure-aware, library API fuzzers.

• Minerva (⭐31) - Minerva is a browser fuzzer augmented by API mod-ref relations, aiming to synthesize highly-relevant browser API invocations in each test case.

• FANS (⭐255) - FANS is a fuzzing tool for fuzzing Android native system services. It contains four components: interface collector, interface model extractor, dependency inferer, and fuzzer engine.

Tools / CPU

• DifuzzRTL (⭐88) - DifuzzRTL is a differential fuzz testing approach for CPU verification.

• MorFuzz (⭐29) - MorFuzz is a generic RISC-V processor fuzzing framework that can efficiently detect software triggerable functional bugs.

• SpecFuzz (⭐30) - SpecFuzz is a tool to enable fuzzing for Spectre vulnerabilities

• Transynther (⭐21) - Transynther automatically generates and tests building blocks for Meltdown attacks with various faults and microcode assists.

Tools / Web

• TEFuzz (⭐16) - TEFuzz is a tailored fuzzing-based framework to facilitate the detection and exploitation of template escape bugs.

• Witcher (⭐91) - Witcher is a web application fuzzer that utilizes mutational fuzzing to explore web applications and fault escalation to detect command and SQL injection vulnerabilities.

• CorbFuzz (⭐3) - CorbFuzz is a state-aware fuzzer for generating as much reponses from a web application as possible without need of setting up database, etc.

Tools / Blockchain

• Fluffy (⭐59) - Fluffy is a multi-transaction differential fuzzer for finding consensus bugs in Ethereum.

• LOKI (⭐14) - LOKI is a blockchain consensus protocol fuzzing framework that detects the consensus memory related and logic bugs.

Tools / DBMS

• Squirrel (⭐197) - Squirrel is a fuzzer for database managment systems (DBMSs).

26. Awesome Games of Coding

Language Independent

• Lambda Spellcrafting Academy - A visual programming / puzzle game where you build spells to solve puzzles. Covers introductory programming, as well as some data structures and algorithms. The language is inspired by Lisp, but the game focuses on programming fundamentals that apply to most languages.

• Prev: Nov 27 - Dec 03, 2023
• Next: Nov 13 - Nov 19, 2023