Searching through Logs is a routinely mission that helps us to retrieve certain critical information,

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

1. Introduction

This project is an intern project Made By Saeed Aburahma the main goal of the project is Optimizing the searching on the logs process by making it Less time Consuming and with side goals that will be shown in the wiki, As it is known Searching through documents or text files is one of the most frequently tasks a user of any Computer might do, the search is arguably one of the most popular tasks. While there exist many different ways to implement a search technique we will try to find the most efficient and effective way to reach our goal.

1.1 Problem Statement

Searching through Logs is a routinely mission that helps us to retrieve certain critical information, At the moment, whenever we have to access the logs to investigate a given issue, we typically have to go through many GBS of data, using zgrep to decompress and search the files, which takes a long time.

In one case, we wanted to search through many 3000 orders logs that zgrep took around 8 days to finish. This isn't ideal, as it hampers our ability to investigate

ideally, we should track most frequently accessed service logs such that when new logs are published they're parsed and indexed based on attributes of interest so when time comes for us to search through the logs, we can search quickly and pinpoint the issue. so the search operation will be more efficient regarding time and more effective to solving problems.

1.2 Requirements

1.2.1 Functional Requirements

1. The system should be able to find and return from its storage, all files relevant to the keyword typed in the search query by a user And at what line the word occurs in the file and the content of that line.

2. The system should return the files relevant to the keyword without taking in consideration sensitivity of the letters.

3. The system should be able to index the files based on a specific pattern that the user enters.

4. The system will be a CLI based.

5. The system should be able while indexing to categorise the logs based on a particular scheme.

6. The system should be able to update the indexed file on a schedule and be able to read files in a chocon time span.

7. The System should be able to trigger an alert via cloud watch if a repeated error was found Screenshot

1.2.2 Non Functional Requirements

1. The system should have low latency. Whenever a user makes a query, we want the search results to appear as fast as possible.

2. The system should be scalable. It should be able to accommodate a growing number of files in its Data Source.

3. The system should be maintainable. it should be easy to change the way of indexing or the searching.

 

Attachments:

Related Questions

. Introgramming & Unix Fall 2018, CRN 44882, Oakland University Homework Assignment 6 - Using Arrays and Functions in C

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

. The standard path finding involves finding the (shortest) path from an origin to a destination, typically on a map. This is an

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

. Develop a program to emulate a purchase transaction at a retail store. This program will have two classes, a LineItem class and a Transaction class. The LineItem class will represent an individual

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

. SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

. Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of